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ABSTRACT 


This  thesis  is  designed  to  act  as  an  instructor’s  supplement  for  refresher  matrix 
algebra  courses  at  the  Naval  Postgraduate  School  (NPS).  The  need  for  a  beginning 
matrix  algebra  supplement  is  driven  by  the  unique  circumstances  of  most  NPS  stu¬ 
dents.  Most  military  students  attend  NPS  several  years  after  receiving  their  under¬ 
graduate  degrees.  This  supplement,  unlike  most  college  textbooks,  bridges  the  gap 
between  the  student’s  educational  lay-off  and  the  rigors  of  mathematically  oriented 
degrees  such  as  applied  math,  operations  research  and  engineering.  By  reviewing 
the  fundamental  concepts  of  vectors  and  matrices,  and  performing  basic  operations 
with  them,  the  student  quickly  develops  the  background  needed  in  NPS’s  demanding 
curriculums.  This  supplement  focuses  on  matrix  and  vector  operations,  linear  trans¬ 
formations,  systems  of  linear  equations,  and  computational  techniques  for  solving 
systems  of  linear  equations.  The  goal  is  to  enhance  current  matrix  algebra  textbooks 
and  help  the  beginning  student  build  a  foundation  for  higher  level  engineering  and 
mathematics  based  courses. 
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INTRODUCTION  TO  MATRIX  ALGEBRA 

Matrix  algebra  is  the  study  of  algebraic  operations  on  matrices  and  of  their 
applications,  primarily  for  solving  systems  of  linear  equations.  Systems  of  linear 
equations  appear  often  in  fields  that  utilize  mathematics  to  solve  real-world  prob¬ 
lems.  Although  most  of  these  real-world  problems  are  so  large  that  they  would  prove 
to  be  too  difficult  and  time  consuming  to  solve  by  hand,  it  is  extremely  important  to 
understand  the  concepts  underlying  their  solution.  Once  the  techniques  for  solving 
small  systems  are  understood,  the  methods  can  be  applied  to  writing  computer  algo¬ 
rithms  used  to  solve  much  larger  systems.  The  purpose  of  this  supplement  therefore, 
is  to  develop  the  student’s  fundamental  skills  of  matrix  algebra  problem  solving  at  a 
very  basic  level.  Where  most  matrix  or  linear  algebra  books  concentrate  on  why,  this 
supplement  concentrates  on  how.  That  is,  the  computational  aspect  of  matrix  alge¬ 
bra  is  stressed  instead  of  the  derivation  of  the  concept.  Besides  algebraic  operations 
on  matrices,  we  also  discuss  linear  transformations.  The  focus  of  the  supplement 
however,  is  to  develop  the  skills  necessary  to  solve  systems  of  linear  equations  using 
several  different  methods.  Whenever  possible,  examples  using  both  real  and  complex 
numbers  are  used  to  get  the  student  ready  for  the  types  of  problems  he  or  she  is  likely 
to  encounter  in  follow  on  classes.  Each  chapter  concludes  with  exercises  designed  to 
develop  the  students  skills  in  the  material  just  covered.  The  solutions  can  be  found 
in  Chapter  Five.  Students  desiring  a  deeper  understanding  of  matrix  algebra  will 
benefit  greatly  from  the  references  used  in  writing  this  supplement. 

Chapter  One  of  the  supplement  deals  with  matrix  notation  and  operations. 
The  student  learns  how  the  elements  of  a  matrix  are  arranged  and  the  difference 
between  square  and  rectangular  matrices.  We  then  introduce  the  basic  algebraic 
operations  for  matrices:  transpose,  conjugate  transpose,  addition,  subtraction,  scalar 
multiplication,  and  matrix  multiplication. 

Chapter  Two  of  the  supplement  is  about  vector  operations  and  their  geomet- 
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ric  representations.  The  student  learns  that  a  vector  can  be  regarded  as  a  matrix 
consisting  of  either  one  row  or  one  column.  Therefore,  the  matrix  operations  of 
transpose,  addition,  and  subtraction  apply  to  vectors  as  well.  Then  we  move  on  to 
vector  multiplication.  Since  a  vector  can  be  thought  of  as  a  matrix,  and  we  know 
how  to  multiply  matrices,  we  can  surely  multiply  a  matrix  times  a  vector.  This  intro¬ 
duces  the  matrix  equation.  The  student  then  learns  that  there  are  two  cases  of  vector 
multiplication-column  x  row  and  row  x  column.  The  dot  product  is  introduced  as 
a  means  of  formally  defining  the  inner-product.  We  then  use  the  inner-product  to 
generalize  terms  associated  in  two  or  three  dimensions,  such  as  length  of  a  vector  and 
the  angle  between  two  vectors.  Using  the  inner-product,  the  student  also  learns  how 
to  project  one  vector  onto  another  and  determine  if  two  vectors  are  orthogonal.  The 
use  of  the  inner-product  allows  us  to  deal  with  both  real  and  complex  vectors.  We 
close  the  chapter  by  showing  how  a  vector  can  be  written  as  a  linear  combination  of 
other  vectors. 

In  Chapter  Three,  the  student  learns  about  linear  transformations  and  how 
certain  structured  matrices  like  the  identity  matrix  or  permutation  matrix  can  trans¬ 
form  one  vector  into  another  vector.  We  then  discuss  matrix  norms.  Next,  we  move 
on  to  range  and  null  space  of  a  matrix.  We  finish  up  the  chapter  discussing  a  special 
type  of  matrix  known  as  the  elementary  matrix.  Elementary  matrices  are  used  to 
develop  methods  for  solving  systems  of  linear  equations  in  the  next  chapter. 

Chapter  Four  covers  systems  of  linear  equations  and  their  solution.  Here  we 
define  what  is  meant  by  a  linear  equation  and  by  a  system  of  linear  equations  and 
demonstrate  several  techniques  used  to  solve  them.  The  first  three  techniques  are  the 
method  of  substitution,  Gaussian  elimination,  and  Gauss-Jordan  elimination.  We 
also  show  the  student  how  to  find  the  inverse  of  a  matrix.  Because  of  their  special 
form,  triangular  matrices  are  then  studied  as  a  lead  in  to  our  last  solution  method, 
the  LU  decomposition.  We  show  two  methods  of  finding  the  LU  decomposition.  We 
finish  the  chapter  by  looking  at  pivoting  as  it  applies  to  computational  efficiency. 
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This  is  a  very  important  concept  when  systems  are  solved  by  computers. 

This  supplement  is  a  compilation  of  several  different  works.  Lecture  notes 
from  both  advisors,  a  similar  matrix  algebra  thesis,  basic  linear  and  matrix  algebra 
books,  and  the  authors  first  hand  experiences  of  what  topics  at  NPS  seemed  to  need 
a  bit  more  refresher  attention. 

The  study  of  matrix  algebra  goes  far  beyond  what  has  been  presented  in  this 
supplement.  The  manual  is  designed  to  give  students  from  varying  backgrounds  the 
necessary  tools  of  matrix  algebra  needed  to  succeed  in  follow-on  courses  being  taught 
at  NPS. 
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PREFACE 


The  writing  of  this  supplement  is  driven  by  the  need  for  a  manual  to  ac¬ 
company  current  introductory  matrix  algebra  textbooks.  It  is  designed  to  provide 
additional  examples  and  to  clarify  difficult  concepts  which  are  presented  in  the  text. 
Students  using  this  supplement  are  assumed  to  have  varied  educational  backgrounds 
with  different  levels  of  knowledge,  although  it  expected  that  all  students  have  a  basic 
facility  with  algebra. 

This  supplement  is  organized  using  the  following  format:  We  first  discuss  a 
concept,  then  formally  define  the  concept,  and  then  provide  examples  showing  its  use. 
Whenever  applicable,  examples  using  both  real  and  complex  numbers  are  used.  At 
the  end  of  each  chapter  there  are  exercises  to  reinforce  the  ideas  that  were  covered  in 
the  chapter.  The  solutions  to  these  exercises  can  be  found  in  Chapter  5. 

Chapter  1  introduces  matrix  notation  and  basic  matrix  operations.  These 
operations  are  performed  using  both  real  and  complex  matrices.  In  Chapter  2  we 
talk  about  vector  notation  and  operations,  linear  combinations,  linear  independence, 
and  spanning  sets.  Chapter  3  is  on  linear  transformations,  including  several  specific 
examples  such  as  the  elementary  matrices.  Chapter  4  focuses  on  methods  for  solving 
systems  of  linear  equations,  these  include  substitution,  Gaussian  and  Gauss-Jordan 
elimination,  matrix  inversion,  and  the  LU  decomposition. 
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MATRIX  OPERATIONS 


We  start  by  thinking  of  a  matrix  as  a  rectangular  table  of  elements.  A  matrix 
composed  of  n  rows  and  m  columns  is  an  element  of  %nXm  if  all  of  its  elements  are 
real  numbers,  and  is  an  element  of  CnXm  if  any  of  its  elements  are  complex  numbers. 
The  complex  set  is  one  that  we  will  deal  with  quite  frequently,  since  /RnXm  q  CnXTO. 
Upper  case,  Roman  letters  will  be  used  to  denote  matrices.  It  is  often  useful  to  refer 
to  a  particular  element  in  a  matrix  by  its  row  and  column  position;  the  element  in 
the  ith  row  and  jth  column  of  a  matrix  A  will  be  denoted  by  or  Ay. 


Example  1.1  A 

columns. 


a\,\  al,2  •  Aim 

a2,l 

L  ‘  ' 


€  Cnxm  is  a  matrix  with  n  rows  and  m 


Example  1.2  Given  A  = 


1  -i  -4 
8  2i  0 

2  2  —  3*  4 

occupying  the  3rd  row  and  2nd  column  position. 


€  C3x3,  03,2  =  2  —  3*  and  is  the  element 


A  rectangular  matrix  that  has  an  equal  number  of  rows  and  columns,  like  the  matrix 
A  in  the  previous  example,  is  called  a  square  matrix. 


A.  MATRIX  TRANSPOSE 

One  of  the  most  useful  operations  that  is  performed  on  matrices  is  transposi¬ 
tion.  This  operation  turns  the  rows  into  columns  and  the  columns  into  rows. 


Definition  1.1  The  transpose  of  a  matrix  A,  denoted  by  AT,  is  defined  by  (A T)y  = 
aji]  that  is,  the  ijth  element  of  AT  is  the  jith.  element  of  A. 


Example  1.3  Given  A  = 


14-60 

3  8  12  3 


compute  At. 


1 


Solution: 


1  3 
4  8 


At 


-6  12 
0  3 


An  operation  related  to  transposition  is  the  construction  of  the  conjugate 
transpose,  or  Hermitian  transpose.  The  conjugate  transpose  of  a  matrix  is  constructed 
by  transposing  the  matrix  and  taking  the  complex  conjugate  of  all  the  elements,  or, 
equivalently,  by  taking  the  complex  conjugate  and  then  transposing. 


Definition  1.2  The  Hermitian  transpose  or  conjugate  transpose  of  a  matrix  A,  de¬ 
noted  by  AH,  is  defined  by  (A H)ij  =  or  equivalently  AH  =  A  =  Ar. 


Example  1.4  Given  A  = 


1  3  i  6 
-i  2  4 


,  compute  At  and  AH . 


Solution: 


1 

—i 

1 

i 

3  i 

2 

and  Ah  = 

-Si 

2 

6 

4 

6 

4 

As  this  example  demonstrates,  the  (Hermitian)  transpose  of  an  n  x  m  matrix  is  an 
m  x  n  matrix.  Note  that  taking  the  (Hermitian)  transpose  of  a  matrix  twice  returns 
the  original  matrix.  That  is,  (AT)T  =  A  and  ( AH)H  =  A. 


Example  1.5  Given  AH  = 


1  i 


-3 i  2 
6  4 


,  compute  (AH)H. 


Solution: 


(Ah)* 


1  Si  6 
-i  2  4 


which  is  our  original  A  from  the  previous  example. 
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Example  1.6  Given  A  = 


1  0  6 
0  2  4 


,  compute  AT  and  AH. 


Solution: 


1  0 

1  0 

at  = 

0  2 

and  Ah  = 

0  2 

6  4 

6  4 

In  the  real  case,  the  transpose  and  the  Hermitian  transpose  are  identical. 


B.  MATRIX  ADDITION,  SUBTRACTION  AND  SCALAR 
MULTIPLICATION 

Matrices  become  useful  when  simple,  algebraic  operations  are  utilized.  When 
matrices  are  the  same  size,  it  is  natural  to  add  and  subtract  them.  This  is  accom¬ 
plished  by  performing  the  operation  element-by-element  on  corresponding  pairs  of 
the  matrix  entries. 

Definition  1.3  Given  A,B  €  CnXm,  the  sum ,  A  ±  B,  is  computed  as 


<*lm  i  b\m 

<*2 to  i  ^2  m 

<* nm  i  bnm 

From  this  definition,  we  see  that  the  resulting  matrix  is  an  n  x  m  matrix,  like  the 
original  matrices,  and  is  an  element  of  Cn/m.  Because  the  reals  are  a  subset  of  the 
complex  numbers,  the  definition  for  addition  and  subtraction  also  applies  to  real 
matrices. 


A  ±  B  = 


<*i,i  i  &i,i 

<*1,2  ±  &i,2 
<*nl  i  ^nl 


Example  1.7  Given  A  = 


1  2 
3  4 


and  B  = 


i  12 
-7  2  i 


compute  A  +  B  and  A  —  B. 
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Solution: 


A  +  B  = 


A-B  = 


1  +  i  2  +  12 

1  +  i 

14 

3-7  4  +  2  i 

-4 

4  +  2  i 

,  and 


1  —  i 

2-12 

i — ‘ 

1 

. 

1 

o 

3  -  (-7) 

4  —  2  i 

10 

4^ 

1 

to 

<s> . 

1 _ 

If  two  matrices  are  not  of  the  same  dimensions,  i.e.,  both  from  'R,nXm  or  CnXm, 
then  these  operations  are  not  defined  and  we  say  the  matrices  are  not  compatibly  sized 
for  addition  and  subtraction. 


Example  1.8  Given  A  = 


1  2 
3  4 


and  B  = 


1  0 
-3  2 
6  4 


,  compute  A  +  B. 


Solution:  A+B  is  not  defined  because  A  and  B  are  not  compatibly  sized  for  addition. 

Another  useful  operation  is  scalar  multiplication,  which  is  defined  as  follows. 

Definition  1.4  Given  a  matrix  A  €  C”XTO  an(j  a  scalar  a  €  C.  The  scalar  product , 
aA,  is  formed  by  multiplying  each  element  of  A  by  a: 


aA 


oca i,i  ao+2  •  ota-ir 

aa2,i 


OtCLnl 


Example  1.9  Given  a  =  3  and  A 
Solution: 

aA  =  3 


1  2 
3  4  i 


aar, 


,  compute  aA. 


1 

2 

3-1 

3-2 

3 

6 

3 

4  i 

CO 

CO 

3-4  i 

9 

12  i 

4 


1  * ' 

—Si  2  and  a  —  —2  +  i,  compute  aA. 

6  4  _ 

Solution: 

-2  +  i  -1-2 * 

«A=  3  +  6i  -4  +  2 i 

— 12  +  6 *  — 8  +  4* 

When  dealing  with  real  numbers  it  is  understood  that  a  +  0  =  a.  Although  we  may 
not  recall  the  exact  definition,  0  is  the  additive  identity.  Yet  what  does  A  +  0  equal? 
Our  definition  for  matrix  addition  dictates  that  we  can  only  add  matrices  that  are 
compatibly  sized.  If  we  make  0  a  matrix  filled  with  all  zeros  and  ensure  it  is  the  same 
size  as  A,  we  see,  A  +  0  =  A. 

'  -2  —Si  ' 

Example  1.11  Given  A  =  —8 i  3  +  4 i  ,  compute  A  +  0. 

8  -  9*  0  J 

Solution: 

-2  -Si  0  0  -2  -Si 

A  +  0  =  —8*  3  +  4*  +  0  0  =  —Si  3  +  4 i 

8  -  9 *  0  0  0  8  -9*  0 

8  —  9*  5  —1  —3* 

Example  1.12  Given  A  =  —2  0  —Si  7  ,  compute  A  +  0. 

3  +  4*  i  6  0  _ 

Solution: 

8  —  9*  5  —1  —3*  0  0  0  0  8  -9*  5 

A  +  0  =  -2  0  -8*  7  +  0000  =  -2  0- 

3  +  4*  i  6  0  0  0  0  0  3  +  4*  i 

"  8  -  9*  5  -1  -Si' 

Example  1.13  Given  A  =  —20—8*  7  ,  compute  A +  1. 

3  +  4*  *  6  0 


Example  1. 10  Given  A  = 


5 


Solution:  A  +  1  is  not  defined,  since  A  and  1  are  not  compatibly  sized  for  addition. 


With  the  definitions  of  matrix  addition,  subtraction,  and  scalar  multiplication, 
we  can  now  list  some  important  properties  of  matrices. 

Theorem  1.1  Given  matrices  A,B,C  €  Cnxm,  and  scalars  a,(3  €  C,  the  following 
properties  of  matrix  addition,  subtraction  and  scalar  multiplication  hold: 


1.  A  +  B  =  B  +  A 

2.  A  +  (B  +  C)  =  (A  +  B)  +  C 

3.  o;(A  +  B)  =  aA  +  <xB 

4.  (a  +  /3) A  =  aA  +  /3A 

5.  (a/2)A  =  /?(aA) 

6.  A+0=  A 

7.  A+  (—A)  =0 


’  <*1,1 

<*1,2 

<*1,3 

'Kr 

b\,2 

b\,z 

Example  1.14  Given  A  = 

<*2,1 

<*2,2 

<*2,3 

,B  = 

&2,1 

^2,2 

^2,3 

<*3,1 

<*3,2 

<*3,3  . 

.  ^3,1 

&3,2 

^3,3  . 

determine  whether  a  (A  +  B)  =  a 

A  + 

a  B. 

and  a  £  C, 


Solution: 


a  (A  +  B) 


a  (ai.i  +  &i,i) 
<*  (<*2,1  +  &2,l) 
<*  (<*3,1  +  &3,l) 


<*  (<*1,2  +  ^1,2) 

<*  (<*2,2  +  ^2,2) 

<*  (<*3,2  +  b 3,2) 


<*  (<*1,3  +  ^1,3) 
<*  (<*2,3  +  ^2,3) 
<*  (<*3,3  +  hz,z) 


oca  1,1  +  a6i,i 
<^<*2,1  +  CK?>2,1 

««3,1  +  <*&3,1 


<*<*1,2  +  <*61 ,2 
<*<*2,2  +  <*&2,2 
<*<*3,2  +  <*&3,2 


<*<*1,3  +  <*&1,3 
<*<*2,3  +  <*^2,3 
<*<*3,3  +  <*&3,3 


<*<*1,1 

aah2 

<*<*1,3 

<*&i,i 

<*&i,2 

<*£>1,3 

<*<*2,1 

oca2l  2 

<*<*2,3 

+ 

<*&2,1 

<*£>2,2 

<*£>2,3 

<*<*3,1 

«a3}2 

<*<*3,3 

<*&3,1 

<*£>3,2 

- 1 

CO 

CO 

rO 

$ 

=  ccA  +  aB. 
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This  example  is  not  a  formal  proof.  However,  we  can  use  this  technique  to  show  that 
the  given  properties  hold,  and  verifications  of  the  remaining  properties  are  left  as 
exercises. 

C.  MATRIX  MULTIPLICATION 

Matrix  multiplication  could  be  defined  in  the  element-wise  manner  of  addition 
and  subtraction,  but  this  leads  to  algebraic  results  of  limited  applicability.  Instead, 
we  define  matrix  multiplication  in  the  following  manner. 

Definition  1.5  Given  A  €  CnXp  and  B  E  CpXm,  the  product  AB  =  C  is  defined  as 

(AB)ij  =  ^2  AikBkj  =  C* j. 

k= 1 


Example  1.15  Given  A 
compute  AB. 


al,l  al,2 
<22,1  <22,2 


€  c2x2  and  B  = 


i>i,i  W,2 
62,1  i>2,2 


€ 


C2X2, 


Solution: 


AB  = 

<2l,l  <2l,2 

&1,1  &1,2 

_ 

<2l,l&l,l  +  <21,2^2,1  <2i,i&i,2  +  <2l,  2&2,2 

<22,1  <22,2 

i>2,l  &2,2 

<22,1^1,1  +  <22,2^2,1  <22,161,2  +  <22,262,2 

Here  we  can  see  that  the  Ci, i  entry  is  the  sum  of  the  products  of  the  elements  of  the 
first  row  of  A  and  the  elements  of  the  first  column  of  B.  The  Ci,2  entry  is  the  sum 
of  the  products  of  the  elements  of  the  1st  row  of  A  and  the  elements  of  the  second 
column  of  B.  The  next  two  entries  of  C,  C2,i  and  02,2,  are  computed  in  the  same 
manner.  Therefore,  we  find  the  ijth  element  of  C  by  finding  the  sum  of  the  products 
of  the  elements  ith  row  of  A  and  the  jth  column  of  B. 


Example  1.16  Given  A  = 


1  3 
-1  2 


and  B  = 


1  -2  0 
1  6  4 


compute  AB. 
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Solution: 


1  -1  +  3- 1 

1(— 2)  +  3-6 

1  -0  +  3-4 

4  16  12 

-1  •  1  +  2-1 

— 1(— 2)  +  2-6 

-1  -0  +  2-4 

1  14  8 

Example  1.17  Given  A  = 


-1  -i 


AB. 


3  +  2?  and  B  = 


1  -2  + 
— i  6 


,  compute 


Solution: 


AB 


(— 1  —  ?)1  +  (3  +  2 (—1  —  t)(— ! 2  +  i)  +  (3  +  2 ?)6 
1—4?  11  +  13*  1- 


In  the  examples  above,  we  found  the  product  AB  by  following  the  simple  rule  of 
“finding  the  sum  of  products  a  row  from  matrix  A  and  a  column  from  matrix  B”.  At 
this  point,  it  should  be  fairly  obvious  that  for  the  matrices  to  be  compatible  under 
this  operation,  the  number  of  columns  in  A  must  equal  the  number  of  rows  in  B. 
These  dimensions  are  called  the  inner  dimensions  of  the  matrix  product.  The  two 
remaining  dimensions  are  called  the  outer  dimensions  as  illustrated  below: 


outer 


(3  x  4)(4  x2) 


Example  1.18  Given  A  €  7£3*4  and  B  €  7 Z4x2,  determine  whether  A  and  B  are 
compatibly  sized  for  matrix  multiplication,  and  if  so,  what  are  the  dimensions  of  the 
resulting  matrix? 

Solution:  The  inner  dimensions  are  equal,  therefore  A  and  B  are  compatibly  sized 
for  matrix  multiplication.  AB  €  7?3x2.  since  the  outer  dimensions  are  3x2. 

Example  1.19  Given  A  £  7£2x2  and  B  €  7l2x3,  determine  whether  A  and  B  are 
compatibly  sized  for  matrix  multiplication,  and  if  so,  what  are  the  dimensions  of  the 
resulting  matrix? 
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Solution:  The  inner  dimensions  are  equal,  therefore  A  and  B  are  compatibly  sized 
for  matrix  multiplication.  AB  €  772x3,  since  the  outer  dimensions  are  2x3. 


Example  1.20 


1 

i 

'  1  2  -i  ' 

Q  A  9 

Given  A  = 

-3  i 

2 

and  B  = 

6 

4  _ 

0  t:  Z 

compute  AB. 


Solution: 


AB  = 


1  i 

-3 *  2 

6  4 


1  2  -i 
3  4  i 


1  +  3i  2  +  4z  — %  —  1 

— 3?  T  6  — 6i  4"  8  — 3  4*  2 i 

6  4-  12  12  4-  16  -6i  +  4i 


1  4-  3i  2  +  4i  — 1  —  i 
6  —  3 i  4-8  —  6i  —3  4-  2 i 
18  28  -2 i 


The  key  factor  that  determines  whether  the  operation  of  matrix  multiplication  is 
possible  is  whether  or  not  the  inner  dimensions  of  A  and  B  are  equal,  or  in  other  words, 
that  the  two  matrices  are  compatibly  sized  for  forming  the  product  AB.  However, 
this  does  not  necessarily  mean  that  they  are  compatibly  sized  for  forming  the  product 
BA.  In  the  previous  example,  if  we  tried  to  find  the  matrix  product  BA,  we  could  not 
since  the  inner  dimensions  of  B  and  A  are  not  equal.  We  say  that  BA  is  not  defined. 
Even  if  the  matrices  are  compatibly  sized,  AB  does  not  necessarily  equal  BA. 


Example  1.21  Given  A  = 
BA. 


and  B  = 


4  5 
9  0 


determine  whether  AB  = 


Solution: 


AB 


BA 


1  -4  +  3-9 

1  -5  +  3-0 

31 

5 

-1  -  4  +  2-9 

-1  -5+2-0 

14 

-5 

4  •  1  +  5(— 1)  4  •  3  +  5  •  2 

-1  22 

9  •  1  +  0(— 1)  9 -3  + 0-2 

9  27 

and 
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Clearly,  these  two  products  are  not  equal  and,  therefore,  matrix  multiplication  as  we 
have  defined  it  does  not,  in  general,  commute. 

Example  1.22  Given  A  €  1Z2x4  and  B  €  7l4x2,AB  €  ft2x2  and  BA  G  7l4x4.  Here 
both  AB  and  BA  exist,  but  the  resulting  matrices  do  not  have  the  same  dimensions. 


Some  important  properties  of  matrix  operations  follow. 

Theorem  1.2  Given  matrices  A,B,C  G  CnXm,  and  scalars  a,  ft  G  C,  the  following 
properties  of  matrix  multiplication  hold,  provided  that  the  indicated  operations  can  be 
performed: 


1.  A(BC)  -  (AB)C 

2.  A(B  +  C)  =  AB  +  AC 

3.  (B  +  C)A  =  BA  +  CA 

4.  a(AB)  =  (aA)B  =  (Aa)B  =  A(c*B)  =  A(Ba)  =  (AB)a 

5.  ( a/3)A  =  a(/3 A) 

6.  (A  +  B)*  =  AH  +  BH 

7.  (AB)*  =  B*A* 


Example  1.23  Given  A  = 

<*1,1  <*1,2  <*1,3 
<*2,1  <*2,2  <*2,3 

,B  = 

61.1  &1,2  &1,3 

62.1  &2,2  &2,3 

<*3,1  <*3,2  <*3,3  . 

.  bz,\  63,2  63,3  _ 

C ,  determine  whether  (c*/?)A  =  <*(/?A). 


and  a,  ft  G 


Solution: 


<*/?<*i,i  apaxa  a/?<*i,3 

£<*1,1  pah2  pah3 

(a/3)A  = 

afia2, 1  apa2, 2  oifia2%3 

—  a 

/?<*2,1  ^<*2,2  /?<*2,3 

a/?<*3,i  ocfia3t2  apa3i3 

/?<*3,1  /?<*3,2  /?<*3,3 

a(/3A). 


This  example  is  not  a  formal  proof.  However,  we  can  use  this  technique  to  verify 
that  property  1.2(5)  holds  and  verifications  of  the  remaining  properties  are  left  as 
exercises. 
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D.  EXERCISES 


3 

1+3* 

2 

i 

to 

i 

cr> 

?S>  . 

7-4* 

2 

1.  Given  A  = 

-6 

,B  = 

-6 

0 

,  and  C  = 

—6  +  4  i 

—5  +  6i 

— 6  +  2  i 

1 

1  —  i 

—5  i 

1 

3-* 

0 

write 

a)  AT  b)  BH  c)  CH 


5  +  i 

1  +  t  -4 

,  and  C  = 

to 

<s> . 

22 

2.  Given  A  = 

7 

,B  = 

1 

05 

1 

Cn 

—3  +  4i 

3  -7* 

-2* 

14  +  —i 

10 

write 


a)  AT 


b)  Bh  c)  Ch 


3 

5 

•  oi 

CN 

I 

i _ 

3-4  * 

7 

0 

3.  Given  A  = 

-6 

,B  = 

-1 

,c  = 

-2* 

0 

,  and  D  = 

1 

00 

1 

2  —  3z 

1 

8 

i 

5 

1  +  5* 

9-6* 

compute 


a)  A  +  B  b)  B  +  D  c)  D  -  C  d)  CH  +  D 
e)  2A  -  3B  f)  iBH  +  2Cr 


5 

i 

4-2 1 

CO 

1 

<>> . 

.  . . i 

4.  Given  A  = 

9 

,B  = 

6 

,c  = 

-2* 

0 

,  and  D 

6 

10 

i 

5 

compute 


i  -2  +  2  *  1 
-8  14*  0 


a)  A-B 


b)  B  +  C  c)  DH  +  C  d)  C*  -  D 


e)  A  +  2B  f)  BH  -  iCH 


5 
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5.  Given  matrices  A  = 
verify  theorem  1.1. 


<*1,1 

<*1,2 

,B  = 

*1,2 

,  and  C  = 

ci,i 

Cl, 2 

<*2,1 

<*2,2 

*2,1 

w 

"io 

i _ 

. C2>i 

<-2,2 

1  2  3 

-1  0  1 

6.  Given  A  = 

4  5  6 

and  B  = 

2  0-3 

7  8  9 

0  2  1 

,  show  AB  ^  BA.  Is  this  true  in 


general? 


-1  0  1 

3  1  0 

2  —  6*  7-4*  2 

7.  Given  A  = 

2  0-3 

,B  = 

0  2  4 

,c  = 

—6  +  4*  —5  +  6*  —6  +  2* 

0  2  1 

1  1  -2 

1  3-*  0 

and  a  £lZ,  compute 

a)  BA  b)  BtC 


:)  C^B 


d)  a(BA) 


e)  BA  +  BC 


8.  Given  A  = 

compute 

a)  AB 


5  4  1 
2  3  2 


,B  = 


2  4  6 

1  2  +  i 

1  2  3 

,c  = 

-3*  3 

1  5  -1 

1-2*  0 

b)  BAT 


:)  C^B 


d)  o(BC) 


,  and  a  €  71, 


e)  AB  +  C 


9.  Given  A  = 
rem  1.2. 


<*i,i 

<*1,2 

,B  = 

’*i,i 

*1,2 

,  and  C  = 

Cl,l 

Cl, 2 

<*2,1 

<*2,2 

*2,1 

*2,2 

C2,l 

C2,2 

,  verify  theo- 
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II. 


VECTOR  OPERATIONS 


We  start  by  thinking  of  a  vector  as  a  special  case  of  a  “thin”  matrix.  Therefore, 
all  of  the  rules  for  matrices  will  also  hold  true  for  vectors.  A  column  vector  is  an 
element  of  7ZnXl.  For  simplicity  of  notation,  we  view  column  vectors  as  elements  of 
7 Zn  if  all  of  the  elements  are  real  numbers,  and  of  Cn  if  any  of  the  elements  are 
complex.  The  superscript  n  denotes  the  number  of  elements  in  the  column  vector. 
Lower  case,  bold  letters  such  as  v  and  w  will  denote  vectors.  Regardless  of  whether  v 
is  a  row  vector  or  a  column  vector,  the  *th  element  of  v  will  be  denoted  by  Vj,  where 
i  denotes  the  position  of  an  element  in  the  vector. 


Definition  II.  1  v 


row  vector. 


Vl 

V2 


is  a  column  vector,  and  w  =  w3 


1- 


wn  is  a 


Example  II.  1  Given  v  = 


1 

2  i 
6 


elements  are  v3  =  6  and  w3  =  —3 i,  respectively. 


,  and  w  =  10  —  3*  —2  +  *  —3*  9  ,  their  third 


Transposition  is  simply  the  act  of  making  a  column  vector  a  row  vector  or  a 
row  vector  a  column  vector.  The  orders  of  the  elements  are  unchanged. 

Definition  II. 2  The  transpose  of  v,  denoted  by  vr,  is  defined  by  (vT),-  =  vp,  that 
is,  the  *th  element  of  the  transpose  of  v  is  the  conjugate  of  the  ith  element  of  v. 


Example  II. 2  Given  w  =  ^  12  0  —3  ,  and  v  = 
Solution: 

12 

and  vT  = 


1 

2 

6 


,  compute  wT  and  vT. 


T 

W  = 


12 

0 

-3 


1  2  6 
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Definition  II. 3  The  conjugate  transpose  of  v,  denoted  by  vH ,  is  defined  by  (v^);  = 
Vi]  that  is,  the  ith  element  of  the  conjugate  transpose  of  v  is  the  conjugate  of  the  ?th 
element  of  v. 


Example  II. 3  Given  v  = 
Solution: 


2  i 
6 


,  compute  vH  and  . 


1  -2 i  6 


and  (v")"  = 


2  i 
6 


Notice  that  transposing  a  vector  twice  returns  the  original  vector.  This  means  that 
{vT)T  =  v  and  ( vH)H  =  v.  Unless  otherwise  noted,  all  vectors  should  be  interpreted 
to  be  column  vectors.  A  row  vector  will  be  written  as  the  transpose  of  a  column 
vector.  In  the  next  few  examples,  remember  to  think  of  vectors  as  skinny  matrices. 

T  r  -1  'j' 

Example  II.4  Given  v  =  J  —2  +  4 i  1  +  3i  ]  and  w  =  \  —6 i  ]  ,  compute 

v  +  w  and  v  —  w. 


Solution: 


V  +  w  = 


v  —  w 


—2  +  1/2  +  4  i 

-3/2  +  4t 

1  +  i( 3  -  6) 

1  -3* 

- 1 

1 

to 

1 

1 — i 

to 

+ 

<s» . 

_ _ i 

-5/2  +  4  i 

1  +  i(  3  +  6) 

1  +  9* 

,  and 


If  two  vectors  are  not  from  the  same  space,  7Zn  or  Cn,  then  the  operations 
of  addition  and  subtraction  are  not  defined  and  we  say  that  the  vectors  are  not 
compatibly  sized  for  addition  and  subtraction. 

Example  II. 5  Given  v  =  —2  +  4i  1  +  3i  4  j  and  w  =  |  — 6z  j  ,  compute 

v  +  w. 

Solution:  v  +  w  is  not  defined,  since  v  and  w  are  not  compatibly  sized  for  addition. 
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Example  II.  6  Given  v  = 
pute  v  —  w. 


-2  0  5  —  3 *  and  w 


-4  3 *  -i  1 


,  com- 


Solution: 


V  —  w  = 


3 

-4 

7 

-2 

3  * 

—2  —  3* 

0 

— * 

* 

•  <s> 
CO 

1 

tO 

1 

•  <s> 

CO 

1 

Example  II. 7  Given  a  =  2  and  v  = 


3  2*  -1  0 


compute  av. 


Solution: 


3  ' 

6  " 

2* 

4* 

-1 

-2 

0 

0 

Example  II.  8  Given  a  =  —  2  +  *  and  v  = 
av. 


-2 i  0 


-1+4* 


-3-5* 


T 

,  compute 


Solution: 

av  =  (—2  +  *) 

To  further  understand  how  these  vector  operations  work,  we  might  look  at 
them  geometrically.  Most  people  have  no  trouble  understanding  vectors  in  the  two- 
dimensional  sense,  as  an  arrow  in  the  Cartesian  plane.  Just  as  we  can  add  and 
subtract  vectors  algebraically,  we  can  also  “add”  and  “subtract”  the  arrows  to  create 
new  arrows,  or  vectors.  The  following  paragraphs  will  attempt  to  tie  together  the 
notion  of  vectors  in  n  space  and  the  geometric  interpretation  of  the  basic  algebraic 
operations  of  addition,  subtraction,  and  scalar  multiplication. 


-2* 

2  +  4* 

0 

0 

-1+4* 

-2-5* 

—3  —  5* 

11  +  7* 
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In  a  simple  two-dimensional  case,  a  vector  can  be  viewed  as  an  arrow  ema¬ 
nating  from  the  origin.  The  elements  of  the  vector  consist  of  the  coordinates  of  the 
head  of  the  arrow,  or,  similarly,  the  distance  moved  horizontally  and  vertically  from 
the  origin.  As  shown  in  Figure  1,  the  vector  v  is  represented  by  the  coordinate  pair 
(3,1),  and  w  by  the  coordinate  pair  (1,3).  We  represent  this  pictorially  as  shown  in 
Figure  1. 


y 


Figure  1 .  Two-Dimensional  Vector  Representation 


T 

* 

v  = 

3  1 

,  and  w  = 

1  3 

This  is  the  two-dimensional  case.  We  can  then  easily  visualize  the  three-dimensional 

r  ~lT 

case.  Given  v  =  \  x  y  z 


,  we  represent  v  by  an  arrow  from  the  origin  to 


?(x,y,z). 

Vectors  in  two  and  three  dimensions  are  easy  to  visualize,  and  we  can  translate  ge¬ 
ometric  concepts  such  as  length  into  vector  operations.  Although  n  dimensions  are 
difficult  to  visualize,  we  can  still  talk  about  addition,  subtraction,  scalar  multipli¬ 
cation,  and  length  of  n- dimensional  vectors.  Notice  again,  that,  as  in  Figure  1,  if 
the  order  of  a  vector  v’s  elements  are  rearranged,  a  new  vector  w  is  formed  that  is 
entirely  different  from  the  original  vector.  So  we  see  that  the  order  of  vector  elements 
is  important. 
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Now  we  look  at  vector  addition  in  the  Cartesian  plane.  To  add  any  two  vectors 
v  and  w  geometrically,  the  vectors  are  first  attached  head  to  tail.  In  Figure  2,  if  we 
add  v  to  w,  the  resulting  vector  v  +  w  is  computed  by  sliding  a  copy  of  vector  w 
without  changing  its  direction,  over  to  the  head  of  vector  v.  Then  draw  a  line  from 
the  origin  to  the  head  of  the  copied  vector  w.  This  new  line  emanating  from  the 
origin  is  the  solution  vector  v  +  w.  If  the  two  vectors  are  added  analytically,  the 
result  will  be  the  same  as  the  geometric  solution.  The  key  to  remember  is  that,  when 
the  copy  of  the  second  vector  is  moved,  its  head  and  tail  must  remain  in  the  same 
relative  positions  as  the  original. 


y 


Figure  2.  Vector  Addition 

Vector  subtraction  is  performed  in  the  same  manner  as  addition.  The  only 
difference  is  that  the  negative  of  the  second  vector  is  used.  Therefore,  as  shown  in 
Figure  3,  to  compute  v  —  w,  we  subtract  vector  w  from  v  by  taking  the  negative  of 
vector  w,  which  reverses  its  direction,  and  then  slide  a  copy  of  it  over  to  the  head  of 
vector  v.  This  is  the  same  procedure  as  adding  two  vectors.  We  are  now  just  adding 
the  negative  of  w. 

To  understand  scalar  multiplication,  it  might  help  to  look  at  Figure  4.  Again, 
we  have  two  vectors  v  and  w.  Notice  v  +  v  =  2v.  Therefore,  scalar  multiplication  is 
just  the  act  of  stretching,  shrinking  or  reversing  the  direction  of  the  original  vector. 
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y 


Figure  3.  Vector  Subtraction 

If  we  multiply  the  vector  v  by  the  scalar  2,  we  see  that  it  is  stretched  to  twice  its 
original  length  in  the  same  direction.  If  we  multiply  vector  w  by  the  scalar  |,  we 
see  that  it  is  stretched  so  that  the  new  vector  is  half  again  as  long  as  the  original 
w.  If  either  of  these  vectors  were  multiplied  by  a  negative  scalar,  the  vector  would 
reverse  direction,  and,  depending  on  the  magnitude  of  the  scalar,  the  vector  would 
be  stretched  or  compressed. 


y 


Figure  4.  Scalar  Multiplication  of  a  Vector 

With  the  definitions  and  geometric  representation  of  vector  addition,  sub¬ 
traction,  and  scalar  multiplication,  we  can  now  show  some  important  properties  of 
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vectors. 


Theorem  II.  1  Given  the  vectors  v,  w,  u  £  Cn,  and  the  scalars  a,  f3  €  C,  the  following 
properties  of  vector  addition,  subtraction,  and  scalar  multiplication  hold: 


1.  v  +  w  =  w  +  v 

2.  v  +  (w  +  u)  =  (v  +  w)  +  u 

3.  a(v  +  w)  =  av  +  aw 

4.  (a:  +  /?)v  =  av  +  /?v 

5.  a(/?v)  =  ( a/3)v 

6.  v  +  0  =  v 

7.  v  +  (-v)  =  0 

8.  lv  =  v 

9.  (-l)v=  -v 

10.  Ov  =  0 


Example  II.  9  Given  v  = 
termine  whether  v  +  w  =  w  +  v. 


V\  v2  v3  v4 


and  w  = 


Wj  w2  iv3  w4 


de- 


Solution: 


V  +  w  = 


Vi  +  W4 

v2  +  w2 
v3  +  w3 
v4  +  w4 


Wi  +  V-L 

w2  +  v2 
w3  +  v3 

W4  +  v4 


=  W  +  V. 


We  can  easily  extend  the  above  example  to  any  size  vector.  The  proofs  of  the  remain¬ 
ing  properties  are  left  as  exercises. 


A.  THREE  SPECIAL  CASES  OF  MATRIX  MULTIPLI¬ 
CATION 

Recall  the  definition  for  matrix  multiplication:  if  A  £  Cnxp  and  B  £  CpXm, 

v 

the  product  AB  is  defined  as  (AB)jj  =  ^  A^Bfcj-  The  first  special  case  occurs  when 
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we  multiply  a  matrix  and  a  column.  The  matrix  multiplication  definition  becomes 
(Ax)  -  =  £  AjfcXfci.  The  equation  Ax  =  b  is  called  the  matrix  equation. 


'  -2  0  ' 

r 

Example  11.10  Given  A  = 

1  4 
-3  5 

and  x  = 

compute  b  =  Ax. 


Solution: 


Ax  = 


-2  0 
1  4 
-3  5 


The  second  special  case  occurs  when  we  left-multiply  a  single  column  by  a  row. 

P 

The  matrix  multiplication  definition  becomes  xTy  =  J2  AifcBfci  =  a.  Notice  that  the 
matrices  must  still  be  compatibly  sized.  The  result  is  a  scalar,  which  leads  to  the 
name  for  this  special  case,  i.e.,  the  scalar  product  (or  a). 


rJT  r  i  y 

Example  11.11  Given  x  =  [  3  —4  0  —2  j  and  y  =  [  4  1  —5  —3  J  ,  com¬ 
pute  xTy. 


Solution:  xT  and  y  are  compatibly  sized  for  matrix  multiplication  since  x  €  7 11*4 
and  y  €  1Z4xi.  Therefore, 


4 

1 

-5 

-3 


=  (12  -4  +  0  +  6)  =  14. 


The  third  special  case  occurs  when  we  left-multiply  a  row  by  a  column.  This 

result  is  called  the  outer  product.  The  matrix  multiplication  definition  becomes  xyT  = 

f)  AnBii  =  C.  Again,  notice  that  the  matrices  must  still  be  compatibly  sized.  So  we 
k= 1 

get  a  square  matrix. 


CT*  r  l  ^ 

Example  11.12  Given  x  =  [  3  -4  0  -2  ]  and  y  =  [  4  1  -5  -3  J  ,  com¬ 
pute  xyr. 
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Solution: 


T 

xy  = 


4 

1 

-5 

-3 


3-40-2 


12 

-16 

0 

-8 

3 

-4 

0 

-2 

-15 

20 

0 

10 

-3 

12 

0 

6 

In  the  last  two  examples  we  computed  the  scalar  product  xTy  and  the  outer  product 
xyT,  using  the  definition  for  matrix  multiplication.  The  results  were  a  scalar  and  a 
matrix,  respectively.  In  the  dot  product  section  we  will  examine  columns  and  rows 
more  carefully. 


B.  VECTOR  NORM 

As  we  have  already  discussed,  we  can  think  of  a  vector  as  a  point  in  ra-space 

where  the  coordinates  of  the  point  are  given  by  the  elements  of  the  vector.  For 

i  T 


example,  we  can  think  of  the  vector  v  = 


3  2 


as  a  point  in  the  Cartesian  plane 


with  coordinates  (3,2).  It  is  also  common  to  think  of  a  vector  as  the  directed  line 
segment  joining  the  origin  and  a  coordinate  point.  Using  the  Pythagorean  theorem, 
we  can  then  find  the  length  of  the  vector,  or  the  distance  from  the  origin  to  the  point. 

Example  11.13  Given  v  =  J  3  2  j  ,  compute  the  length  of  v. 


Solution:  The  length  of  v  is  \/32  +  22  =  y/l3. 

Notice  that  this  looks  similar  to  the  special  case  of  a  row  times  a  column, 


T 

V  V  = 


3  2 


3 

2 


=  32  +  22  =  13. 


All  we  have  to  do  is  take  the  square  root  of  this  special  case  and  we  have  the  length. 

Definition  II. 4  Given  a  point  (x,y)  in  the  plane,  the  distance  from  the  origin  to  the 
point  is  \/x2  -f  y2. 
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Since  vTv  is  the  square  of  the  length  of  v,  it  seems  natural  to  define  the  length  of  a 
vector  using  this  special  case. 


Definition  II. 5  Given  a  vector  v  €  Cn ,  the  Euclidean  length,  or  2-norm,  of  v,  de¬ 
noted  by  ||  v||2  ,  is 


r  1T  n 

Example  11.14  Given  v  =  — 3  1  4  j  ,  compute  ||v 


Solution: 


Example  11.15  Given  v  =  c  _j_  and  a">  c,  d  €  7Z,  compute  vHv. 


Solution: 


r  ]  a  +  m 

vHv  =  a -hi  c-di  =  (a  -  bi)(a  +  hi)  +  (c  -  di)(c  +  di) 

■  ■*  c  +  di 


=  a2  —  b2i2  +  c2 


d2i2  =  a2  b2  +  c2  +  d2 . 


This  operation  always  produces  a  real  scalar,  and  therefore  agrees  with  our  notion  of 
length. 

T 

Example  11.16  Given  v  =  [  —2  +  4 i  1  +  3 i  5  —3  —  i  ,  compute  ||v||2  . 


Solution: 


/ 


Vv^v  = 


— 2  —  4i  1  —  3i  5  —3  +  i 


\ 


-2  +  4  i 


\~2 


1+3  * 


5 

—3  —  i 


) 


=  V20  +  10  +  25  +  10  =  a/65. 


It  is  important  to  understand  there  are  numerous  other  norms  which  are  all  abstrac¬ 
tions  of  the  notion  of  length.  One  of  these  norms  is  the  p-norm.  The  2-norm  is  a 
specific  case  (p  =  2)  of  a  p-norm,  which  is  defined  as  follows. 

Definition  II. 6  Given  a  vector  v  (E  Cn,  the  p-norm  of  v,  denoted  by  ||v||  ,  is 


Ml p  =  \  J2  kfj  >  where  p  ^  L 


Example  11.17  Given  v  = 


Solution: 


-215-3 


,  compute  Hvllj . 


IMIi  —  l_2|  +  |1|  +  |5|  +  | — 3|  —  11. 
Example  11.18  Given  v  =  —2  1  5  —3]  compute  |jv||2  . 


Solution: 


|v||j  =  (|-2|2  +  |1|2  +  |5|2  +  [— 3[2)4  =  V39. 


Example  11.19  Given  v  = 


-215-3 


»  compute  ||v|L  • 


Solution: 


VIL=  max  |  Vi  |=max{|— 2|,|1|,|5|,|-3|}  =  5. 


We  found  || v by  letting  p  — »•  oo  in 


Cs1-1') 


and  taking  the  limit. 
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C.  THE  DOT  PRODUCT 

Vector  addition  and  subtraction  in  the  plane  seem  natural,  and  we  can  visualize 
how  a  solution  is  found.  On  the  other  hand,  we  can  define  the  notion  of  vector 
multiplication  in  many  different  ways.  The  dot  product,  also  known  as  the  scalar 
product,  is  one  way  to  define  vector  multiplication. 

Definition  II.7  Given  two  vectors  v,  w  G  7 Zn,  the  dot  product  of  v  with  w,  denoted 
by  v  •  w,  is 


- 

* 

Vi 

W\ 

V2 

• 

w2 

Vn 

Wn 

=  (virwi  +  v2w2  4 - vnwn). 


Example  11.20  Given  v  = 


-4  1 


r  it 

and  w  =  2  —6  ,  compute 


V  •  w. 


Solution: 

V  •  w  =  (— 4)(2)  +  (l)(-6)  =  -8  -  6  =  -14. 

Note  that,  when  we  take  the  dot  product  of  two  vectors,  the  result  is  a  scalar,  hence 
the  name  scalar  product.  The  definition  of  dot  product  is  valid  for  real  vectors,  but 
we  will  need  a  different  definition  in  order  to  deal  with  complex  vectors.  The  dot 
product  is  a  specific  case  of  an  inner  product.  The  next  definition  is  also  a  special 
case  of  an  inner  product.  We  will  use  this  definition  by  default  when  we  are  talking 
about  the  inner  product. 

Definition  II. 8  Given  two  vectors  v,w  G  7in,  the  inner  product  of  v  with  w,  de¬ 
noted  by  vTw,  is 
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Vi  V2  •  •  vn 


=  (vittfi  +  v2w2  H - 1-  vnwn ). 


T 

V  w  = 


w2 


[  wn  J 

This  is  the  dot  product,  again.  But  this  definition  is  a  natural  extension  of  the 
definition  of  matrix  multiplication.  The  inner  product  for  complex  vectors  is  similar 
to  that  of  the  reals,  except  that  the  Hermitian  transpose  of  v  is  used  instead  of  the 
transpose  of  v,  as  shown  in  the  following  definition. 


Definition  II. 9  Given  two  vectors  v,  w  E  Cn,  the  inner  product  of  w  with  v,  denoted 
by  w-^v,  is  given  by 


H 

W  V  — 


Wi  W2 


Vl 

V2 


(uitOi  +  v2w2  H - f-  vnwn). 


v 


n 


Example  11.21  Given  v 
wHv. 


-2  +  4  i 


1  +  3i 


and 


w 


compute 


Solution: 


wHv 


2  i 


-2  +  4  i 
1  +  3  i 


(2)(— 2  +  4i)  +  (0(1  +  30 


—  — 4  +  8?  +  i  —  3  —  — 7  +  9z. 

What  happens  in  this  example  if  we  reverse  the  order  of  the  vectors?  That  is,  if 
we  compute  vHw?  If  the  vectors  are  real,  we  get  the  same  result.  But  notice  what 
happens  when  the  vectors  are  complex,  as  in  the  following  example. 


25 


r  -iT  .  iT 

Example  11.22  Given  v  =  -2  +  4 *  1  +  3*  and  w  =  [  2  -*  j  ,  compute 

vHw. 

Solution: 

1 1 

V  w 

=  -4  -  8*  -  i  -  3  =  -7  -  9*. 

This  is  the  complex  conjugate  of  the  product  in  the  previous  example,  and  demon¬ 
strates  that,  in  general,  vHw  ^  wHv.  However,  wHv  =  vHw  always  holds.  If  one 
or  both  vectors  are  complex,  then  the  two  different  orders  of  computing  the  inner 
product  lead  to  results  that  are  complex  conjugates  of  each  other.  Notice  that  the 
definition  for  the  inner  product  of  complex  vectors  can  be  used  for  real  vectors. 

Example  11.23  Given  v  =  [  -2  +  4 i  1+3*  5  ]T  and  w  =  [  2  —i  ]  ,  compute 
wrv. 

Solution:  wTv  is  not  defined  since  the  two  vectors  are  not  compatibly  sized  for 
vector  multiplication. 

r  +  r  -iT 

Example  11.24  Given  v  =  —2  +  4*  1  +  3*  5  J  and  w  =  [  —5*  -4  —  *  6  j  , 

compute  wHv. 

Solution: 

XJ 

W  V 

=  5*( — 2  +  4*)  +  ( — 4  +  *)  (1  +  3*)  +  (6)  (5)  =  3  —  21*. 


-2  +  4* 

5*  —4  +  *  6  1+3* 

5 
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Now  that  we  have  defined  the  inner  product,  we  can  show  some  properties  of 
vector  products. 

Theorem  II. 2  Given  the  vectors  v,  w,  u  6  Cn ,  and  the  scalars  a,  /3  €  C,  the  following 
properties  of  vector  multiplication  hold: 

1.  vT(w  +  u)  =  vTw  +  vTu  (left  distributive  law) 

2.  (wT  +  uT)v  =  wrv  +  uTv  (right  distributive  law) 

3.  a(vT w)  =  (avT)w  =  vT(cxw) 

The  proofs  are  left  as  exercises. 


D.  ORTHOGONALITY 

Orthogonality  is  another  term  best  explained  in  a  specific  and  easily  visualized 
case.  In  7Z2  and  V?  we  say  that  two  vectors  v  and  w  are  orthogonal  if  they  are 
perpendicular,  and  write  vlw.  As  the  dimensions  of  v  and  w  increase,  it  is  difficult 
to  visualize  perpendicular  vectors.  If  we  talk  about  complex  vectors  it  is  nearly 
impossible  to  visualize  perpendicular  vectors.  However,  we  can  define  “perpendicular” 
vectors  of  any  dimension,  real  or  complex.  We  will  use  the  inner  product  to  determine 
whether  two  vectors  satisfy  our  definition. 


Definition  11.10  Vectors  v,  w  6  Cn  are  orthogonal  if  vHw  —  0. 


Example  11.25  Given  v  =  2  4  J  and  w  =  2  —  1  j  ,  determine  whether  v 

and  w  are  orthogonal. 


Solution: 


vTw  =  (2)(2)  +  (4)(-1)  =  0. 


Therefore,  v  is  orthogonal  to  w.  This  can  be  quickly  verified  by  drawing  the  vectors 


in  the  Cartesian  plane.  If  we  look  at  Figure  5,  we  see  that  v  = 
iT 


2  4 


tow  = 


-1 


since  they  are  separated  by  90  degrees.  Yet,  u  =' 


is  orthogonal 
-1  3  ]  is  not 


orthogonal  to  either  v  or  w.  This  can  be  verified  by  computing  the  inner  product  or 
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Figure  5.  Vector  Orthogonality 

by  looking  at  the  vectors  geometrically.  Although  it  is  difficult  to  visualize  more  than 
three  dimensions,  using  the  definition  of  orthogonality  we  can  determine  whether  two 
vectors  having  the  same  number  of  elements  are  “perpendicular”  or  not. 

Example  II.26  Given  v  =  J  —2  1  5  —3  j  and  w  =  [  —5  —4  7  6  j  ,  deter¬ 
mine  whether  v  and  w  are  orthogonal. 


Solution:  vrw  =  —  2(-5)  +  1( — 4)  +  5(7)  -  3(6)  =  23.  Therefore,  v  and  w  are  not 
orthogonal. 


Example  11.27  Given  v  = 
determine  whether  v  and  w 


4  0  2  4  — 7  ]T  and  w 
are  orthogonal. 


—2  9  6  —1  0 


Solution:  vTw  =  4(— 2)  +  0(9)  +  2(6)  +  4(— 1)  -  7(0)  =  0.  Therefore,  v  and  w  are 
orthogonal. 


Example  11.28  Given  v  = 
determine  whether  v  and  w 


-1-4*  8*  1  ]' 
are  orthogonal. 


and  w  = 


6  -1  —  1  +  32i 


Solution:  v^w  =  (—1  +4«)(6)  —  8i(— 1)  +  2(0)  =  —1  +  32i.  Therefore,  v  and  w  are 
not  orthogonal. 
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and  w  =  3  —  1  3  +  2i  ,  de- 


Example  11.29  Given  v  =  [  —  1  +  2i  8i  1 
termine  whether  v  and  w  are  orthogonal. 

Solution:  vHw  =  (—1  +  2z)(3)  +  8«( — 1 )  +  1(3  +  2 i)  =  0.  Therefore,  v  and  w  are 
orthogonal. 


E.  PROJECTIONS 

Before  beginning  the  discussion  of  projections,  let’s  first  define  a  unit  vector 
and  what  is  meant  when  we  refer  to  the  angle  between  two  vectors.  If  u  (E  Cn  and 
||u||  =  1,  then  u  is  a  unit  vector. 

Definition  II.  11  Given  any  vector  x  €  Cn,  a  unit  vector  u  can  be  formed  by  dividing 
the  vector  x  by  its  magnitude  or  2-norm: 


x 


The  proof  that  shows  why  u  is  a  unit  vector  is  straightforward.  We  start  by  using 
the  formula  for  calculating  a  unit  vector,  u  =  pjj,  then  we  find  the  magnitude  of  u. 
(If  u  is  a  unit  vector,  its  magnitude  must  be  equal  to  1).  Happily, 


u||  = 


X 

ix  ~i 

U 

f  T  \ 

XJX 


,XTX, 


as  anticipated.  This  argument  also  holds  true  for  complex  vectors.  Remember  that 
the  magnitude  or  2-norm  of  a  vector  is  equal  to  the  length  of  the  vector.  Therefore, 
when  we  divide  a  vector  by  its  magnitude,  the  result  is  a  vector  with  unit  length, 
pointing  in  the  direction  of  the  original  vector. 


Example  11.30  Given  v  = 


4  0  2  4 


—7  ,  form  the  unit  vector  u. 


Solution: 


v  _  1 

\ZVTv  \/89 


4 


T 


0  2  4  -7 
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Example  11.31  Given  v  = 
u. 


-2  +  4i  1  +  3*  5  -3- 


,  form  the  unit  vector 


Solution: 


v  v  1 

|v  |  a/v^v  \/65 


-2  +  4i 


1  +  3*  5 


— 3  —  * 


Figure  6.  Angle  Between  Two  Vectors 

Next,  let’s  define  the  angle  6  between  two  vectors  in  II2,  as  shown  in  Figure  6.  Starting 
in  the  plane  and  using  the  law  of  cosines,  we  can  find  the  angle  between  two  arbitrary 
arrows.  Since  the  arrows  in  the  plane  correspond  to  vectors,  all  we  have  to  do  is  use 
the  law  of  cosines  on  the  vectors  and  clean  up  the  notation.  In  Figure  6,  the  arrows  a 
and  b  are  connected  by  the  arrow  a  —  b  =  c.  The  angle  between  the  arrows  a  and  b 
is  0.  The  lengths  of  the  arrows  are  ||a||  =  a,  ||b||  =  b,  and  ||c|[  =  c.  The  law  of  cosines 
states  c2  =  a2  +  b2  —  2abcos0.  Making  the  vector  substitutions,  a  =  v,  b  =  w,  and 
c  =  v  —  w  and  using  the  law  of  cosines,  we  have 

II  V  -  w  ||2  =||  v  ||2+  ||  w  ||2  -  2  ||  v  ||  ||  w  ||  cos  0. 

Now  expanding  ||v  —  w||2  ,  we  have 

||  v  —  w  ||2  =  (v  —  w)  •  (v  —  w)  =||  v  ||2  —  2v  •  w+  ||  w  ||2. 
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Equating  these  two  equations  yields 


||  v  ||2  -  2v  •  w+  ||  w  ||2=||  v  ||2+  ||  w  ||2  -2  ||  v  ||  ||  w  ||  cos  6. 
Canceling  like  terms  leaves  us  with 

— 2v  •  w  =  —  2  ||  v  ||  ||  w  ||  cos  9. 


Finally, 


vHw  =||  v  ||  ||  w  ||  cos  6. 


Theorem  II. 3  Given  vectors  v  and  w  in  7 Z2,  the  angle  0  between  v  and  w  is  defined 
by  vHw  =  ||v||  || w||  cos  0.  This  generalizes  to  Cn. 


This  fact  is  useful  when  we  are  given  two  vectors  and  desire  to  find  the  angle  6  between 
them. 


Example  11.32  Given  v  = 
pute  9. 


-215-3 


and  w  =  —5—476], 


com- 


Solution: 


cos  0  = 


V  •  w 


V  w 


-!=—} — =■  Therefore,  0  —  cos  1(  r^f=). 


lT 


Now  let’s  make  a  couple  of  observations  about  the  vector  v  =  1  1  j  m 

Figure  7.  The  length  of  v  is  ||  v  ||  =  \/2.  Also  notice  the  angle  that  v  makes  with  the 
x  axis  is  j  radians  or  45°.  Applying  simple  geometry  we  find  that  the  length  of  the 
y  coordinate  is  -\/2sin  =  1  and  the  length  of  the  x  coordinate  is  \/2cos  =  1. 

This  gives  us  the  components  of  v  along  the  y  and  x  axes  respectively.  This  can 
be  restated  as  the  projection  of  v  onto  the  y  and  x  axes.  The  projection  vectors 


are  represented  by 


IT 


0  1 


and 


-lT 


1  0 


respectively.  However,  we  can  find  the 


components  of  v  with  respect  to  any  arbitrary  vector,  not  just  the  coordinate  axes. 
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Figure  7.  Length  of  a  Vector 

Definition  11.12  The  projection  ofv  onto  w,  denoted  by  proj^v,  is  the  portion  of 
w  which  is  constructed  by  passing  vector  z  perpendicular  to  w  to  the  end  of  v.  The 
projwv  extends  from  the  origin  to  the  base  of  z. 

Definition  11.13  Given  vectors  v,  and  w,  the  magnitude  of  projwv  is  given  by 

||projwv||  =  ||v||  cos  0. 


y 


Figure  8.  Projection  Vector 


In  Figure  8,  we  draw  a  vector  z  perpendicular  to  w  which  passes  through  the  end  of 
v.  In  order  to  find  projwv  =  p  as  in  Figure  8,  all  we  have  to  do  is  multiply  ||projwv|| 
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by  a  unit  vector  in  the  direction  of  w.  To  find  p,  let’s  first  look  at  our  definition  in 

T* 

vector  form.  Replacing  cos  6  in  the  previous  definition  with  'j|v|[n|w'|i  we  ^ave 

w^v 

\\projwv\\  =  ||v| 


V 


w 


Canceling  like  terms  we  obtain  ||projwv||  —  Finally  multiply  ||  proj^v  ||  by  a 
unit  vector  in  the  direction  of  w: 
w 


w 


lb^iwv|| 


w  WTV 


W  w 


=  w 


WTV 

w^w 


=  proj  wv. 


Definition  11.14  Given  vectors  v,  and  w,  the  projection  ofv  onto  w  is 

T 

W  V 

projwv  =  w  =  . 

wJ  w 

This  gives  us  the  component  of  v  in  the  direction  of  w. 

Example  11.33  Given  v  r  lT 

pute  projwV. 

Solution: 


-2  1  5  -3  ]  and  w  =  [  -5  -4  7  6  ]  , 


com- 


T 

W  V 

proj-wV  =  w  —  = 

w1  w 


-5 

-4 

7 

6 


23 

126' 


Example  11.34  Given  v  =  [  4  0  2  4  — 7  ]  and  w  =  [  — 2  9  6  —1  0 
compute  projwv. 


Solution: 


T 

W1  V 

proj  wv  =  w  = 

wJ  w 


Notice  this  agrees  with  what  we  found  earlier  about  the  orthogonality  of  v  and  w. 


1 

1 

- 1 

O 

- 1 

0 

0 

0 

2 

= 

0 

122 

4 

0 

1 - 

1 

- 1 

o 

_ 1 
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Example  11.35  Given  v  = 
projwv. 


-1  -4  i 


8i  2 


T 


and  w  = 


1  T 

6—10  ,  compute 


Solution:  From  a  previous  example,  we  know  that  vHw  =  —  1  +  32i.  Therefore, 


proj-wV  = 


6 

-1 

0 


-1  -  32 i 

V37 


In  the  derivation  of  the  definition  of  projection,  we  used  vector  notation.  As  we 
have  said,  there  is  very  little  difference  between  vectors  and  arrows  in  the  plane.  The 
advantage  is  that  in  vector  notation  we  can  leave  the  plane  and  enter  n-dimensional 
spaces  and  maintain  our  geometric  interpretation.  But,  as  you  may  have  already 
noticed,  we  will  continue  to  use  the  two-dimensional  and  three-dimensional  language 
even  when  we  are  in  n-dimensional  space. 


F.  LINEAR  COMBINATIONS 

Now  that  we  have  the  ability  to  perform  addition,  subtraction,  and  scalar 
multiplication  on  vectors,  we  can  build  weighted  sums  of  vectors,  called  linear  combi¬ 
nations. 


Definition  11.15  Given  vectors  V!,v2  •  •  •  V/  6  Cn,  and  scalars  ai,a2  •  •  •  on,  the 

i 

weighted  sum  w  =  J2  ai vi  =  <*ivi  +  <*2v2  +  •  •  •  +  cqv/  is  a  linear  combination  of  the 

i= 1 

vectors  Vi,  v2  •  •  •  V/  and  is  also  called  the  vector  equation. 


Example  11.36  Given  Vj  = 


1  2 


n  r  1  T 

and  v2  =  2  4  ,  find  scalars  oj  and  a2 


such  that  w  =  4  8]  is  a  linear  combination  of  Vi  and  v2. 


Solution:  Remember,  to  be  a  linear  combination  of  Vi  and  v2,  w  must  be  repre¬ 
sentable  as  a  weighted  sum  of  vectors  Vi  and  v2.  Therefore,  we  must  find  aj  and  a2 
such  that 
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4 

1 

2 

= 

+  Oi2 

8 

2 

4 

By  inspection,  if  we  let  cti  =  2,  and  a2  =  1,  then 


i 

=  2 

1 

+  1 

2 

1 - 

OO 

i 

CN 

_ i 

i 

1 _ 

So  we  see  that  there  is  at  least  one  set  of  scalars  for  which  w  is  a  linear  combination  of 
Vj  and  v2.  Later  it  will  become  much  clearer  how  to  find  these  scalars  mathematically. 
The  notion  of  a  linear  combination  is  one  of  the  most  fundamental  ideas  in  linear 
algebra  and  it  will  come  up  over  and  over  again  throughout  this  text. 


'  1  ‘ 

’  0  ' 

'  0  ' 

0 

1 

0 

Vi  = 

0 

,v2  = 

0 

,v3  = 

1 

.  0  . 

.  0  . 

.  0  . 

Example  11.37  Given  v4  = 


can  form  linear  combinations  of  vl5  v2,  v3,  and  v4  as  follows: 


and  v4  = 


0 

0 

0 

1 


we 


— 2v4  +  3v2  +  7v3  —  4v4 


1  t 


-237-4 


20vi  H — 6v2  +  0v3  +  9v4  = 


and 


-it 


20  -6  0  9 


Example  11.38  Given  v4 
form  linear  combinations  of  Vi,v2,v3,and  v4  as  follows: 


i  ' 

‘  0  ' 

‘  0  ' 

‘  0  ' 

0 

i 

0 

,  and  v4  = 

0 

0 

,v2  = 

0 

»v3  = 

i 

0 

.  0  . 

.  0  . 

.  0  . 

i  . 

,we  can 


— 2va  +  3v2  +  7v3  —  4v4  = 


—2 i  3 i  7 i  —4 i 


and 


20v4  4 — 6v2  +  0v3  +  9v4  — 


1  t 


20 i  —6 i  0  9 i 
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G.  LINEAR  INDEPENDENCE 


We  say  a  set  of  vectors  is  linearly  independent  if  no  vector  of  the  set  can  be 
written  as  a  linear  combination  of  the  other  vectors  of  the  set. 


Definition  IL16  Given  a  set  of  vectors  {vi,V2, . ,V/},  a  homogeneous  equation  is 

an  equation  of  the  form: 


0  =  ajVi  +  a2V2  H - h  a/V/. 


Definition  11.17  If  the  only  solution  to  the  homogeneous  equation  is  the  trivial 

solution,  that  is  c*i  =  a2  =  •••  =  <*/  =  0,then  the  set  {vl5  v2, . ,  V/}  is  said  to  be 

linearly  independent.  If  a  nontrivial  solution  exists,  then  the  set  of  vectors  is  said  to 
be  linearly  dependent. 

Example  11.39  Given  vectors  w,  v,  and  u,  determine  whether  they  are  linearly  in¬ 
dependent. 


Solution: 


4 

1 

2 

w  = 

8 

,v  = 

2 

,  and  u  = 

4 

0 

-1 

2 

By  inspection,  we  see  that  the  following  homogeneous  equation  has  a  nontrivial  so¬ 
lution.  Let  =  —1,  ct2  =  2,  and  0:3  =  1. 


0 

4 

1 

2 

0 

0 

=  -1 

8 

+  2 

2 

+  1 

4 

= 

0 

0 

0 

-1 

2 

0 

Therefore,  w,  v,  and  u  are  linearly  dependent. 


H.  SPANNING  SETS 

In  a  previous  section  we  saw  how  a  set  of  vectors  could  be  used  to  form  new 
vectors.  Here  we  will  use  the  notion  of  linear  combinations  to  define  the  span.  The 
span  is  the  set  of  all  possible  linear  combinations  of  a  given  set  of  vectors.  This  will 
also  lead  to  the  idea  of  vector  equations. 
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Definition  11.18  Given  a  set  of  vectors  S  =  {vj,  v2,  •  •  •,  v/},  the  span  of  S ,  denoted 
by  span{vi,  V2,  •  •  •,  Vj},or  span(S'),  is  the  set  of  all  linear  combinations  of  the  vectors 
in  S.  If  V  C  span(5),  then  S  is  also  known  as  a  spanning  set  of  V. 


Example  11.40  Given  v  = 


0  ■ 

1  ' 

'  -2  ‘ 

-1  +  * 

2  —  3* 

,  w  = 

2  i 

4  —  7  i 

,  and  b  = 

1 

1 

3  . 

0  . 

i  . 

mine  whether  b  is  in  span{v,  w}. 


deter- 


Solution:  In  other  words,  can  we  find  two  complex  scalars,  ctj,  and  02,  such  that 


0  ’ 

1 

-2 

-1  +  * 

2* 

1 

aa 

2  —  3* 

+  oc2 

4  —  7* 

1 

3 

0 

i 

In  order  to  find  a  solution  we  must  solve  four  equations: 


<*1  (0)  +  a2  (1) 

oq  (-1  +  i)  +  a2  (2 i) 
cci  (2  —  3i)  +  0.2  (4  —  7 i) 
<*i  (3)  +  a2  (0) 


-2 

1 

1 

i. 


However,  <*1,  and  0.2  must  satisfy  all  four  equations  simultaneously.  By  inspection, 
we  see  there  are  no  values  for  ai  and  a2  that  satisfy  all  four  equations  simultaneously. 
This  tells  us  that  b  is  not  a  linear  combination  of  v  and  w.  Therefore,  b  is  not  in  the 
span  of  v  and  w.  Keep  this  example  in  mind  when  we  move  into  systems  of  linear 
equations. 


I.  BLOCK  OPERATIONS 

It  is  often  convenient  to  think  of  a  matrix  as  having  elements  that  are  them¬ 
selves  matrices.  By  doing  this,  one  can  impose  structure  on  a  matrix,  expose  the 
structure  of  a  matrix,  or  somehow  simplify  the  way  we  look  at  the  matrix.  For  very 
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large  matrices,  the  concept  of  block  matrices  is  a  must.  Matrices  whose  elements 
are  themselves  matrices  are  called  block  matrices.  In  a  sense,  every  matrix  is  a  block 
matrix  if  you  think  of  its  elements  as  1  x  1  matrices  instead  of  scalars.  However,  there 
are  many  other  ways  of  partitioning  a  matrix  into  blocks. 

'  1  2  3  ‘ 

Example  11.41  Given  A  =  4  5  6  , 


‘  i  1  2  ' 

vi  =  4  ,  v2  =  5  ,  v3 

7  8 


2  ,u2 

3 


5  ,  and  u3  =  8 


two  additional  ways  of  representing  A  in  the  form  of  a  block  matrix  are 


vi  v2  v3  and  A  — 


Two  block  matrices  are  compatible  for  addition  if  they  have  the  same  number 
of  rows  and  columns  and  the  individual  blocks  are  compatibly  sized  for  addition.  Two 
block  matrices  are  compatibly  sized  for  multiplication  if  the  first  matrix  has  as  many 
columns  as  the  second  matrix  has  rows  and  the  number  of  columns  in  each  column 
block  of  the  first  is  the  same  as  the  number  of  rows  in  the  corresponding  row  block 
of  the  second. 

Example  11.42  Given  u  =  [  uf  uj  uj  ]  and  v  =  [  va  v2  v3  ]  ,  compute  uTv 
and  vuT. 


Solution: 


ufVi  ufv2  ufv3 


Vi  v2  v3  =  u^Vi  ujv2  ujv3  ,  and 


u3  Vi  ujv2  ujv3 


Vj  v2  v3 


T 


+v2u^+v3u| 
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The  preceding  example  shows  the  outer  product  form  of  matrix  multiplication 
and  the  inner-product  form  of  matrix  multiplication,  respectively.  By  looking  at  these 
two  forms,  we  can  see  that  the  inner  product  form  of  multiplying  two  n  x  n  matrices 
requires  the  computation  of  n 2  inner  products  at  a  cost  of  n  multiplications  and  n  —  1 
additions  each,  for  a  total  of  n3  multiplications  and  n3  —  n2  additions.  The  outer- 
product  form  requires  the  computation  of  n  outer  products  with  n2  multiplications 
each  for  n3  multiplications,  and  n  —  1  additions  of  nxn  matrices  for  n3  —  n2  additions. 
That  is,  the  same  number  of  multiplications  and  additions  are  required.  However, 
if  we  time  the  two  forms  of  matrix  multiplication  in  MATLAB  with  two  100  x  100 
matrices,  we  will  find  that  it  takes  roughly  55%  longer  to  do  a  matrix  multiply  using 
the  inner  product  form.  There  are  many  other  ways  of  partitioning  matrices  into 
blocks  that  we  will  see  as  they  become  useful. 

J.  EXERCISES 


—  1  - 1—  2i 

-2 

1.  Given  x  = 

2-3  i 

,y  = 

3 

,  and  z  = 

i  —2  +  4i  6 

-7  i 

1 

pute 

a)  xH 

b)  yT 

c)  x  +  y  d)  3x  -  2iy 

e)  x  +  z  f)  x  +  zT  g)  2zh  h)  z  +  z 


write  or  corn- 


Ui 

V\ 

Wi 

2.  Given  u  = 

U2 

,V  = 

Vi 

,  and  w  = 

w2 

,  verify  Theorem  II.  1  parts  2 

U3 

V3 

w3 

through  10. 
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3.  Given  x  = 

1 

2 

,  y  = 

-3 

0 

,  and  z  = 

5 

2 

,  compute 

a)  x  -  z 

4 

b)  x 

2 

•y 

c)  x(z  • 

-2 

y) 

d)  (y  • 

-5 

-2 

-5 

4.  Given  x  = 

2 

,y  = 

3 

,  and  z  = 

7 

,  compute 

1 

1 

0 

a)  x  •  z  b)  (x  •  z)y  c)  x(z  •  y)  d)  (y  •  z)x 


5.  Given  u  = 


u2 


v  — 


V2 


Uz 


Vz 


W 1 


and  w  = 


w2 


w3 


verify  Theorem  II. 2. 


—5  +  i 

-2 

—5  —  2  i 

6.  Given  x  = 

2-3  i 

,y  = 

3-* 

,  and  z  = 

-4* 

,  compute 

1+4  i 

1  -f*  Si 

3 

a)  xTy 

b)  yT: 

z  c) 

T 

zJx 

d)  yTx 

e)  zTy 

f)  xrz 

7.  Using  x,y,  and  z  from  question  4,  compute 

a)  (xrz)  y  b)  x  (zry)  c)  (yTx)  z  d)  y  (xrz) 


8.  Using  through  x,y,  and  z  from  question  6,  compute 
a)  (xHz)  y  b)  x  (zHy)  c)  (y^x)  z 


d)  y  (xHz) 
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9.  Given  w  = 


,  find  all  the 


1 

4 

0 

2 

-2 

,x  = 

-2 

,y  = 

1 

,  and  z  = 

-2 

1 

2 

2 

-6 

orthogonal  pairs. 


4 

6 

1 

-2 

1 

2 

10.  Given  x  = 

0 

,y  = 

-3 

,  and  z  = 

5 

2 

4 

0  _ 

a)  Compute  ||x|| . 

b)  Compute  ||y|| . 

c)  Compute  ||z|| . 

d)  Are  x  and  y  orthogonal? 

e)  Are  z  and  y  orthogonal? 

f)  Are  x  and  z  orthogonal? 


4  +  2z 

-2  i 

1 

1 1 .  Given  x  = 

-2 

>y  = 

l+i 

,  and  z  = 

2  i 

— 3  % 

-Si 

-1  +  4  i 

—5  i 

-1  +  lOi 

i 

a)  Compute  ||x|| . 

b)  Compute  ||y|| . 

c)  Compute  ||z|| . 

d)  Are  x  and  y  orthogonal? 

e)  Are  z  and  y  orthogonal? 

f)  Are  x  and  z  orthogonal? 
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-4 

1 

l — 1 
05 

_ 1 

2 

6 

12.  Given  v  = 

-5 

,w  = 

3 

7 

-2 

compute 


a)  ||v||  b)  ||w||  c)  ||x|| 


-2 

3 

0 

-5 

13.  Given  v  = 

3 

and  w  = 

-4 

1 

2 

—i 

6 

X  = 

2  —  i 

— 3  +  2 1 

,  and  y  = 

-3  -  4 * 

—5  * 

6  +  5  *  _ 

2 

d)  |Jy ||  e)  Find  all  the  orthogonal  pairs 


compute 


a)  wTv 

b)  wTw 

c)  cos0,  where  6  is  the  angle  between  v  and  w. 

d)  projwv 


-2 

3 

14.  Given  v  = 

3 

and  w  = 

-5 

3 

-4 

-2 

0 

,  compute 


a)  wrv 

b)  wTw 

c)  cos 0,  where  6  is  the  angle  between  v  and  w. 

d)  projwv 
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15.  Given  v  = 


,  compute 


-2 1 

3  +  i 

-2 

3 

3 

,w  = 

-5  i 

,x  = 

0 

,  and  y  = 

-5 

3  -i 

-4 

3 

—4 

-2  +  4  i 

0  _ 

1 

2 

cos#,  where  6  is  the  angle  between 

a)  w  and  v  b)  w  and  x  c)  x  and  y  d)  v  and  y  e)  x  and  v 
f)  w  and  y 


16.  Given  vectors  u,  v,w,x  €  773,  and  the  vector  equation  — 2(2x  +  v)  =  3(w  —  u), 
express  x  as  a  linear  combination  of  u,v,  and  w. 


17.  Given  v  = 

i 

1 

1 _ 

,w  = 

-2* 

i 

,X  = 

-2 

3 

,  and  y  = 

2 

—i 

3  -i 

2  -j-  2  % 

1 

-3  i 

a)  Write  a  vector  equation  with  w,  x,  and  y  as  independent  variables  multiplied 
by  some  scalar,  and  v  as  the  dependent  variable. 

b)  Find  w  +  x  +  y.  Is  v  in  the  span  {w,  x,  y}? 

c)  Are  v,w,x,  and  y  linearly  independent? 


1  -3  -2 

6 

18.  Given  A  = 

-2  2  6 

and  b  = 

-3 

2  -8  -3 

5 

columns  of  A? 

is  b  a  linear  combination  of  the 
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2 

-1 

3 

4 

9 

1 

0 

-2 

7 

11 

19.  Given  A  = 

3 

-3 

1 

5 

and  b  - 

8 

2 

1 

4 

4 

_  10  _ 

columns  of  A? 


b  a  linear  combination  of  the 
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III.  LINEAR  TRANSFORMATIONS 


Linear  transformations  are  extremely  powerful  tools.  In  reality,  a  linear  trans¬ 
formation  is  simply  a  way  of  looking  at  what  goes  into,  and  what  comes  out  of  a 
black  box.  When  speaking  about  functions  of  the  form  y  =  f(x),  we  take  a  variable 
x,  perform  some  function  on  it,  and  get  a  result  y.  The  function  is  the  black  box, 
with  input  x  and  output  y.  We  use  special  terms  to  define  ideas  about  this  system. 
The  function  f(x)  is  a  “black  box”,  which  assigns  y  (an  element  from  the  range)  to 
x  (an  element  from  the  domain).  In  other  words,  x  is  transformed  by  the  function 
/(x)  into  y.  When  discussing  linear  transformations  involving  vectors  and  matrices, 
we  will  use  special  terms  to  define  similar  ideas.  Recall  that  a  function  is  a  rule  that 
assigns  to  each  element  in  a  set  A  (the  domain),  one  and  only  one  element  from  a 
set  B  (the  codomain).  When  dealing  with  vectors  and  matrices  we  will  use  the  terms 
map  or  transform  instead  of  assign.  Therefore  a  linear  transformation  (“black  box”) 
will  map  (or  transform)  a  vector  x  to  a  vector  b.  Given  a  function  /,  the  notation 
/  :  A  — >  B  means  that  /  is  a  transformation  from  A  to  B. 

Now  let’s  make  the  jump  to  linear  algebra  and  see  that  transformations  are 
functions. 

Definition  III.l  A  transformation  from  %n  to  %m  is  a  rule  that  assigns  to  each 
vector  x  £  Hn  a  vector  y  £  7Zm .  Formally,  we  say  that  a  transformation  T,  denoted 
by  T  :Kn  — ►  Km,  maps  7ln  into  71™. 

We  can  use  just  about  any  rule  to  define  a  transformation.  In  matrix  algebra,  though, 
we  are  particularly  interested  in  transformations  of  the  form  T(x)  =  Ax.  Such  a 
transformation  takes  a  vector  x,  and  transforms  it  by  multiplying  on  the  left  by  a 
compatibly-sized  matrix  A.  Since  we  know  that  a  vector  x  multiplied  by  a  matrix  A  is 
some  vector  b,  then  we  can  write  Ax  =  b,  or  in  other  words,  a  matrix  A  transforms  a 
vector  x  into  a  vector  b.  Suppose  that  x  £  7Zn  is  mapped  to  a  vector  b  £  7Zm  by  the 
matrix  A.  Based  on  our  knowledge  of  matrix  multiplication,  what  can  we  say  about 
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the  dimensions  of  A?  Since  x  6  7Zn  and  b  €  7Zm  then  A  €  7ZmXn.  Now  that  we  know 
what  a  transformation  is,  let’s  define  exactly  what  is  meant  by  linear  transformation. 

Definition  III. 2  T  :  7 Zn  — >  7 Zm  is  a  linear  transformation  if,  for  any  vectors  x  and 
y  €  7Zn  and  any  scalar  c  €  71,  the  following  two  conditions  hold: 

1)  T(x  +  y)  =  T(x)  +  T(y) 

2)  T(cx)  =  cT(x). 

This  definition  implies  that  it  doesn’t  matter  whether  addition  or  scalar  multiplication 
is  performed  before  or  after  the  transformation;  the  same  results  will  be  achieved.  In 
order  to  clarify  this  idea,  we  will  use  the  analogy  of  a  chef  in  the  kitchen  preparing 
a  salad  and  baking  bread.  Generally,  the  act  of  chopping  is  a  linear  transformation. 
You  can  either  chop  the  vegetables  up  individually,  then  mix  them,  or  you  can  mix  the 
whole  vegetables  and  then  chop  the  entire  mixture.  Either  way,  you  will  end  up  with  a 
bowl  of  chopped  vegetables.  However,  baking  in  general  is  not  a  linear  transformation. 
A  good  chef  will  follow  the  baking  instructions  and  mix  the  ingredients  first,  then 
bake  the  bread.  A  skeptical  mathematician  will  bake  the  ingredients  individually  and 
then  try  to  mix  the  ingredients.  Surely,  bread  can’t  be  made  this  way.  The  order  of 
baking  and  mixing  is  very  important. 

Example  III.l  Given  T(x)  =  Ax,  determine  whether  T  is  a  linear  transformation. 

Solution:  In  order  to  determine  this,  T  must  satisfy  the  two  conditions  previously 
stated.  We  start  with  the  first  condition,  T(x  +  y)  =  T(x)  +  T( y). 

Working  with  the  left  side  of  the  equality  by  definition  of  T,  we  have 

T(x  +  y)  =  A(x  +  y). 

Since  matrix  multiplication  is  distributive,  it  follows  that 

A(x  +  y)  =  Ax  +  Ay. 
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Now,  working  with  the  right-hand  side  of  the  equality, 


T(x)  +  T(y)  =  Ax  +  Ay. 


Therefore, 

T(x  +  y)  =  r(x)  +  r(y). 

This  satisfies  the  first  condition,  but  what  about  the  second  condition?  We  must 
show  that 

T(cx)  =  cT(x). 

Working  with  the  left-hand  side  of  the  equality,  we  have 

T(cx)  =  A(cx). 

Since  scalar  multiplication  of  vectors  and  matrices  is  commutative,  then 

A  (ex)  =  cAx. 

Now  working  with  the  right-hand  side  of  the  equality,  we  have 

cT(x)  =  cAx. 

Therefore,  T(cx)  =  cT(x),  which  satisfies  the  second  condition.  Since  conditions  one 
and  two  are  satisfied,  T(x)  =  Ax  is  a  linear  transformation.  If  either  condition  did 
not  hold,  then  the  transformation  would  not  have  been  linear. 


Example  III. 2  Given  T(x)  =  s^n(^)  _ 

V' x2 

whether  T  is  a  linear  transformation. 


xi 

X2 


,  and  y  = 


yi 

2/2 


,  determine 


Solution:  Again  starting  with  condition  one, 


T(x  +  y)  =  T 


®i  +  2/i 
Z2  +  2/2 


sin(x!  +yi) 
\/X2  +  2/2 


,  while 
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r(x)+T(y)  = 


T(x  +  y) 


sin(xi) 
y/x 2 


+ 


sin(yi) 

y/V2 


sin(xi)  +  sin(yi) 
\f% 2  +  v^2 


,  so 


sin(xi  +  2/i ) 
+  3/2 


7^ 


sin(xi)  +  sin(i/i) 
\Z+2  +  y/V2 


r(x)  +  T(y). 


Because  condition  one  does  not  hold,  T  is  not  a  linear  transformation. 


Example  III.3  Given  T(x)  = 


2zi 

£2 


,x  = 


£2 


,  and  y  = 


2/i 

V2 


,  determine 


whether  T  is  a  linear  transformation. 


Solution:  Again  starting  with  condition  one,  we  have 


T(x  +  y) 


(2-1  +  2/1 
£2  +  2/2 


2(xi  +  2/i ) 
£2  +  2/2 


T(x)  +  T(y)  = 

2x\ 

+ 

2yi 

— 

2(£i  +  2/1) 

£2 

2/2 

£2  +  2/2 

Condition  one  holds,  but  we  must  still  check  condition  two: 


T(cx) 


=  T 

/ 

c 

£i 

\ 

_ 

c2o:i 

=  c 

2xi 

.  X 2  . 

) 

CX  i 

£i 

and 


cT  (x)  -  cT 


( 

£i 

\ 

2x\ 

=  c 

V 

£i 

) 

Xl 

Because  condition  two  also  holds,  we  say  that  T  is  a  linear  transformation. 

In  engineering  and  physics,  one  of  the  most  useful  tools  is  the  simplification 
of  a  problem.  By  splitting  a  problem  up  into  smaller  problems,  large,  seemingly 
unsolvable  problems  become  a  number  of  smaller  solvable  problems.  The  solutions  to 
the  smaller  problems  can  then  be  combined  to  get  a  solution  to  the  larger  problem. 
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This  process  is  based  on  the  superposition  principle  and  is  a  generalization  of  our  two 
conditions: 

T(clXl  +  c2x2  +  •  •  •  +  cnxn)  =  c1T(x1)  +  c2T(x2)  +  •  •  •  +  cnT(xn). 

A.  SPECIAL  MATRICES 

There  are  a  number  of  matrices  that,  because  of  their  frequency  of  use  and 
importance,  have  been  given  special  names.  These  special  matrices  have  specific  useful 
properties.  We  start  with  perhaps  the  most  important  special  matrix  of  all. 

1.  The  Identity  Matrix 

The  first  special  matrix,  and  probably  the  most  widely  used,  is  the  identity 
matrix.  The  identity  matrix  performs  the  same  function  as  the  number  1  in  multipli¬ 
cation;  la  =  a  and  bl  =  b.  This  matrix  is  denoted  by  I,  and  has  ones  along  the  main 
diagonal  and  zeros  elsewhere.  (The  main  diagonal  elements  of  a  matrix  A  are  those 
elements  that  are  indexed  by  a a.) 

Example  III.4  The  matrix  I  G  7 Z4x4  is  an  identity  matrix. 

1  0  0  0  " 

°  1  0  0 
0  0  10 
0  0  0  1 

When  dealing  with  square  matrices,  the  identity  matrix  is  special  because  IA  =  AI  = 
A  This  is  a  special  case  in  which  matrix  multiplication  commutes. 

Example  III.  5  Given  A  = 
pute  IA  and  AI. 


i 

8 


,  and  the  identity  matrix  I 


1  0 
0  1 


,  com- 
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Solution: 


IA 


AI 


1  0 

3  i 

1  *  3  -h  0( — 1)  1  •  i  0  •  8 

3  i 

0  1 

-1  8 

0  *  3  +  1( — 1)  0  •  i  +  1  ■  8 

-1  8 

3  i 

1  0 

3-1+z-O 

3  -  0  +  2  - 1 

3  i 

1  8 

0  1 

F 

1 

1 - 1 

+ 

oo 

o 

—  1  (0)  4-8-1 

-1  8 

Notice  that  IA  =  AI.  In  most  cases,  the  dimensions  of  the  identity  matrix  are  inferred 
from  the  context  of  the  associated  matrices. 


2.  Permutation  Matrices 

A  second  very  important  special  matrix  is  a  permutation  matrix.  This  square 
matrix  is  denoted  by  P,  and  is  very  closely  related  to  the  identity  matrix.  PA  trans¬ 
forms  A  by  exchanging  the  rows  of  A.  AP  exchanges  the  columns  of  A.  The  per¬ 
mutation  matrix  transforms  a  matrix  from  'JZmxn  to  7 Zmxn  and  a  vector  from  7Zn  to 
Hn. 


Example  III. 6  Given  P  = 
Solution: 

'  PA  = 

P  exchanges  the  rows  of  A. 


0  1 
1  0 


and  A  = 


2  6* 
1  -3 


,  compute  PA. 


0  1 

2  6  i 

1 - 

CO 

1 

r*H 

1  0 

1 

CO 

1 

i-H 

_ 1 

2  6  i 

Example  III. 7  Given  P 
pute  AP. 


‘  1 

0 

0  ' 

5 

— 10  +  8  i 

0  ‘ 

0 

0 

1 

and  A  = 

— 

2 

0 

1 

0 

l-i 

1 

-3 

com- 


Solution: 


AP  = 


.5-10  +  8*  0 

1  0  0 

- 1 

Ot 

o 

1 

1 — * 

o 

+ 

00 

CO  . 

_ I 

—5  i  2  6i 

0  0  1 

— 

—5  i  6  i  2 

l-i  1-3 

0  1  0 

l-i  -3  1 
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P  exchanges  the  columns  of  A. 

An  important  concept  to  grasp  is  that  certain  rows  of  matrix  A  can  remain 
in  the  same  position  when  computing  PA.  This  is  done  by  placing  a  one  in  any  such 
row’s  main  diagonal  position  of  matrix  P.  Additionally,  certain  columns  of  A  can 
remain  in  the  same  position  when  computing  AP.  This  is  done  by  placing  a  one  in 
any  such  column’s  main  diagonal  position  of  matrix  P. 


Example  III.  8  Given  P  = 
PA. 


'  0 

0 

1  ■ 

'  0 

3 

4  ' 

0 

1 

0 

and  A  = 

-5 

6 

1 

1 

0 

0 

i 

2  i 

3 

,  compute  matrix 


Solution: 


0  0  1 

0  3  4 

i  2i  3 

0  1  0 

-5  6  1 

= 

-5  6  1 

1  0  0 

i  2  i  3 

0  3  4 

Since  the  1  in  the  second  row  of  P  occupies  the  main  diagonal  position,  row  two  of  A 


remains  fixed,  while  rows  one  and  three  are  interchanged. 


3.  Orthogonal  Projection  Matrices 

An  orthogonal  projection  matrix  can  find  the  components  of  a  vector  v  along 
any  given  axis.  The  orthogonal  projection  matrix  transforms  a  vector  from  1Zn  to 

nn. 


Example  III.  9  Given  Q  = 


1  0 
0  0 


and  v 


x  y 


compute  Qv. 


Solution:  Qv  is  the  component  of  v  which  lies  on  the  x  axis: 


Qv  = 


1 

0 

X 

X 

0 

0 

.  y  . 

0 

Q  takes  v  and  projects  it  onto  the  x  axis. 
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Example  III.  10  Given  Q  = 


x  y  z 


,  compute  Qv. 


Solution: 


1 

0 

0 


0  0  ' 
1  0 
0  0 


and  v  = 


T 


1 

0 

0 

X 

X 

Qv  = 

0 

1 

0 

y 

= 

y 

0 

0 

0 

z 

0 

Q  projects  v  onto  the  {x.  y )  plane.  If  we  continue  with  a  4  x  4  orthogonal  projection 
matrix,  we  could  see  what  a  four-dimensional  object  would  look  like  if  we  projected 
the  four- dimensional  object  into  three  dimensions.  This  is  the  same  idea  of  drawing 
a  three-dimensional  box  on  a  two-dimensional  piece  of  paper. 


4.  Rotation  Matrices 

A  rotation  matrix  transforms  a  vector  v  by  rotating  it  about  the  origin  through 
an  angle  of  9  degrees.  Its  derivation  goes  beyond  the  scope  of  this  text,  but  its  utility 
merits  an  example.  A  rotation  matrix  transforms  a  vector  from  7 Zn  to  7Zn. 


Tv. 


\  cos  6 

—  sin  0 

’ll' 

i - 

CO 

S' 

cos  6 

’  V  “ 

,  and  6  —  j,  compute 


Solution: 


COS  7 

4 

-sinf 

1 

0 

sin  ~ 

4 

cosf  _ 

1 

_V2_ 

T  applied  to  v  rotates  v  about  the  origin  counter-clockwise  by  ^  radians,  or  45°. 


B.  MATRIX  NORMS 

Although  there  are  many  different  matrix  norms,  we  are  interested  in  a  family 
of  norms  called  p-norms.  When  we  found  ||b|| ,  we  called  it  the  length.  The  next 
question  is,  what  is  |[ A || ?  This  is  not  an  easy  question  to  answer.  First  let  us  look  at 
Ax.  When  one  forms  the  product  Ax,  where  A  €  CnXTn  and  x  €  Cmxl,  the  result  is  a 
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vector  b  €  Cnxl  or  Ax  =  b.  In  some  sense,  A  transforms  the  x  into  a  new  b  that  is 
a  linear  combination  of  the  columns  of  A.  It  is  useful  to  know  how  the  norm  of  Ax, 
denoted  ||Ax|| ,  relates  to  the  norm  of  x,  denoted  ||x|| . 


Definition  III. 3  Given  x  €  7£m,  and  A  €  7£nXTO ,  the  p-norm  of  A,  denoted  by  ||A||p, 

llAxll 

is  defined  by  max  ||A||p  =  p ,  where  p  represents  a  specific  vector  norm. 

The  most  common  of  these  norms  are  the  1,  2,  and  oo  norms,  denoted  by  ||A||j ,  ||A||2 
and  IIAH^,  respectively.  In  effect,  the  norm  of  a  matrix  tells  us  just  how  much 
a  matrix  can  stretch  vectors.  If  ||A||  <  1  then  we  know  that  the  matrix  A  can 
transforms  vectors  x  into  shorter  vectors  b,  i.e.,  A  shrinks  vectors.  If  ||A||  >  1, 
then  we  know  the  matrix  A  transforms  some  vectors  x  into  longer  vectors  b,  i.e., 
A  stretches  some  vectors.  The  calculation  of  the  matrix  norm  varies  depending  on 
which  norm  is  desired.  The  easiest  p-norm  to  find  is  the  oo-  norm: 

or  equivalently,  the  maximum  absolute  row  sum.  The  1-norm  is  also  easy  to  calculate. 

l|A|ll=l“a<Xn{^|afcil} 

or  equivalently,  the  maximum  absolute  column  sum. 


Example  III.  12  Given  A  = 


-2  1 

1  -2 

0  1 


01 


,  compute  HAIIj  and  HA^. 


Solution:  || Ajjj  is  the  maximum  absolute  column  sum. 


l|A||j  —  max{|— 2|  +  |1|  +  0  —  3,  |1|  +  |—  2|  -f  |1|  —  4,  0  +  |1|  +  |— 2|  —  3}  . 


The  maximum  value  of  the  absolute  column  sum  is  4,  therefore,  ||  A||j  =  4.  This  result 
indicates  that  Ax  might  stretch  x  to  as  much  as  4  times  its  length.  The  oo-norm  is 
the  maximum  absolute  row  sum: 


|| A || Qj,  —  max {|— 2 1  +  |l|  +  0  —  3,  |1|  +  |  — 2]  +  |1|  —  4, 0  +  |1|  +  |  — 2 1  =  3}  . 
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The  maximum  absolute  row  sum  is  4,  therefore,  ||  AU^  =  4.  This  result  also  indicates 
that  Ax  might  stretch  x. 

Due  to  the  difficulty  in  computing  the  2-norm,  we  leave  it  for  another  class. 

C.  THE  RANGE  AND  NULL  SPACE  OF  A  MATRIX 

In  this  section  we  introduce  two  important  concepts  in  linear  algebra. 

Definition  III. 4  Given  a  matrix  A  €  Cnxm ,  the  range  of  A,  denoted  by  R(A),  is 
given  by  R(A)  =  {y  €  Cn  |  y  =  Ax  for  some  x  G  Cmj. 

In  effect,  R(A)  is  the  set  of  all  linear  combinations  of  the  columns  of  A,  and  so  is 
sometimes  called  the  column  space.  The  null  space  is  related  to  the  column  space. 
Using  the  columns  of  A,  we  form  a  homogeneous  equation, 


Any  x  that  satisfies  the  homogeneous  equation  is  in  N(A). 

Definition  III. 5  Given  a  matrix  A  €  CnXm,  the  null  space  of  A,  denoted  by  N(A), 
is  given  by  N(A)  =  {x  6  Cn  |  Ax  =  0}. 

T  —2  6  —4  1 

Example  III.13  Given  A  =  ^  _3  _2  ,  compute  N(A). 

Solution:  Form  the  homogeneous  vector  equation: 


By  inspection,  we  see  that  any  x  =  t  t  t  j •  ,  where  t  €  11,  is  transformed  into 
the  zero  vector. 
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D.  RANK 


We  have  already  defined  linear  independence  of  vectors.  In  this  section  we  will 
think  of  the  columns  of  a  matrix  as  individual  vectors,  and  then  determine  the  linear 
independence  or  linear  dependence  of  the  columns.  This  enables  us  to  determine  the 
rank  of  a  matrix.  We  start  this  section  with  an  example,  to  review  what  is  meant 
when  we  say  that  the  columns  of  a  matrix  are  linearly  independent  or  dependent. 


Example  III.  14  Given  A  = 
are  linearly  independent. 


'  -1  0  0  1  ' 

-110  0 
-10  10 
-10  0  1 


determine  whether  the  columns  of  A 


Solution:  We  start  by  letting  the  columns  of  the  matrix  be  the  individual  vectors 


-1 

0 

0 

1 

-1 

1 

0 

,  and  a4  = 

0 

aj  = 

-1 

,a2  = 

0 

,a3  = 

1 

0 

-1 

0 

0 

1 

Now  we  ask  whether  vectors  ai,  a2,  a3,  and  a4  are  linearly  independent  or  dependent. 
Using  what  we  learned  earlier,  we  form  a  linear  combination  of  the  vectors  into  a 
homogeneous  equation  Cxaj  +  c2a2  +  c3a3  +  c4a4  =  0.  If  the  only  solution  to  this 
equation  is  the  trivial  solution,  scalars  ci,c2,c3,  and  c4  =  0,  then  the  vectors  are 
linearly  independent.  If  a  nontrivial  solution  exists,  then  the  vectors  are  linearly 
dependent.  At  this  time  we  will  determine  the  values  for  c  by  inspection,  and  ignore 
the  methods  for  computing  them.  We  leave  that  to  later  sections.  If  we  rearrange  the 
homogeneous  equation  into  ciai  +  c2a2  +  c3a3  =  —  c4a4  and  set  all  of  the  constants 
to  1,  we  have  ai  +  a2  +  a3  =  — a4.  Substituting  the  vectors  into  the  homogeneous 
equation  yields 
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-1  1  0  1  0  1  -1 

-1  10  0 

+  + 

-10  1  0 

-l  J  o  J  o  J  -l 

Since  — a 14  can  be  written  as  a  linear  combination  of  a1;a2,  and  a3,  the  columns  of 
A  are  linearly  dependent.  This  looks  like  the  homogeneous  matrix  equation,  except 
that  we  have  moved  the  fourth  column  to  the  right-hand  side  of  the  equation.  We 
could  have  instead  moved  the  first  column  to  the  right-hand  side  of  the  equality. 

0  ]  [  0  1  1 

10  0  1 

+  +  “ 

0  1  0  1 

0  J  0  J  1 J  [1 

Again  we  see  that  the  columns  of  A  are  linearly  dependent,  because  a  nontrivial 
solution  to  the  homogeneous  equation  exists:  C\  =  c2  =  C3  =  C4  =  1.  Since  the  columns 
of  A  can  be  thought  of  as  vectors,  and  since  the  vectors  are  linearly  dependent,  then 
the  columns  of  A  are  linear  dependent.  Notice  that  three  vectors  are  necessary  to 
form  a  linear  combination  of  the  fourth  vector.  This  is  not  a  coincidence.  If  we  take 
any  three  columns  of  A  and  form  the  homogeneous  equation  C\  ax  +  c2a2  +  c3a3  =  0, 
we  will  find  that  the  three  columns  are  linearly  independent. 

Now  that  we  have  reviewed  what  is  meant  by  linear  dependence  and  indepen¬ 
dence,  we  can  move  on  to  the  idea  of  rank. 

Definition  III. 6  The  column  rank  of  a  matrix  is  equal  to  the  number  of  linearly 
independent  columns.  The  row  rank  of  a  matrix  is  equal  to  the  number  of  linearly 
independent  rows.  The  column  rank  is  equal  to  the  row  rank  and  is  called  the  rank 
of  a  matrix.  If  the  number  of  linearly  independent  columns  or  rows  is  equal  to  the 
smallest  dimension  of  the  matrix,  then  the  matrix  is  said  to  have  full  rank. 

Theorem  III.l  Given  A  €  7 ZnXm,x  €  7 Z,  and  T(x )  =  Ax,  then  the  following  are 
true: 
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a)  rank(A)  =  r 

b)  r  <  n  and  r  <  m 

c)  A  has  r  linearly  independent  columns 

d)  A  has  r  linearly  independent  rows 

e)  rank(A)  =  rank(AT) 

f)  The  linearly  independent  columns  of  A  span  R( A) 

g)  The  linearly  independent  rows  of  A  span  R( AT) 


Example  III.  15  Given  A  = 
pendent  rows  of  A. 


10  0  1 
0  10  1 
0  0  11 


determine  the  number  of  linearly  inde- 


Solution:  Notice  that  the  fourth  column  is  a  linear  combination  of  the  first  three 
columns;  i.e.  col4  =  coll  +  col2  +  col3.  Therefore,  A  has  three  linearly  independent 
columns.  This  implies  that  rank(A)  —  3,  and  so  A  also  has  three  linearly  independent 
rows. 


Example  III.  16  Given  A  = 
pendent  columns  of  A. 


'  1 
1 
2 
0 


0  1  ' 
1  0 
1  1 
-1  1 


determine  the  number  of  linearly  inde- 


Solution:  Notice  that  the  third  row  is  a  linear  combination  of  the  first  two  rows; 
i.e.  row3  =  rowl  +  row2.  Also,  the  fourth  row  is  a  linear  combination  of  the  first 
two  rows;  i.e.  row4  =  rowl  —  row2.  Therefore,  A  has  two  linearly  independent  rows. 


Example  III.  17  Given  A  = 


and  so 

A 

also  1 

1 

5 

9 

13  ' 

2 

6 

10 

14 

3 

7 

11 

15 

4 

8 

12 

16 

,  determine  rank(A). 
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Solution:  In  this  example,  it  is  doubtful  whether  the  reader  could  come  up  with  a 
solution  by  inspection.  Other  methods  are  required  to  solve  this  problem.  Therefore, 
at  some  point,  we  must  shift  our  attention  from  the  concept  of  linear  independence 
to  the  algebraic  details  of  finding  a  nontrivial  solution  to  the  homogeneous  equation. 
We  promise  to  discuss  these  details  later. 

E.  ELEMENTARY  MATRICES 

Elementary  matrices  are  a  family  of  special  matrices  that  can  be  used  to 
perform  specific  operations  on  matrices.  Elementary  matrices  will  enable  us  to  find 
new  ways  to  solve  systems  of  linear  equations.  Each  elementary  matrix  E  performs  one 
row  operation  on  a  matrix.  This  operation  is  known  as  an  elementary  row  operation. 
There  are  three  elementary  row  operations:  row  interchange ,  row  replacement,  and 
scalar  multiplication  of  a  row.  We  form  an  elementary  matrix  by  applying  one  of  the 
row  operations  to  the  identity  matrix. 

1.  Row  Interchange 

A  row  interchange,  as  its  name  suggests,  occurs  when  any  two  rows  of  a  matrix 
are  interchanged. 


10  0  0 
0  10  0 
0  0  10 
0  0  0  1 

changes  the  second  and  fourth  rows  of  a  matrix. 


Example  III.  18  Given  I  = 


,  form  an  elementary  matrix  which  inter- 


Solution:  Interchange  the  second  and  fourth  rows  of  I  to  form  the  elementary  matrix 


E  = 


10  0  0 
0  0  0  1 
0  0  10 
0  10  0 
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Using  our  newly  formed  elementary  matrix  E,  we  can  now  interchange  the  second  and 
fourth  rows  of  any  4  x  m  matrix  by  left-multiplying  that  matrix  by  E.  As  we  learned 
previously,  E  is  a  permutation  matrix. 


Example  III.  19  Given  A 
EA. 

Solution:  . 


■  1 

2 

3 

4  ' 

'  1 

0 

0 

0  ‘ 

5 

6 

7 

8 

and  E  = 

0 

0 

0 

1 

9 

10 

11 

12 

0 

0 

1 

0 

.  13 

14 

15 

16  . 

.  0 

1 

0 

0  . 

EA  = 


1 

0 

0 

0 

1 

2 

3 

4 

1 

2 

3 

4 

0 

0 

0 

1 

5 

6 

7 

8 

13 

14 

15 

16 

0 

0 

1 

0 

9 

10 

11 

12 

9 

10 

11 

12 

0 

1 

0 

0  _ 

13 

14 

15 

16  . 

5 

6 

7 

8  _ 

Notice  that  E  interchanges  the  second  and  fourth  rows  of  A. 

,  compute  EA. 


a  b 
c  d 


and  E  = 


0  1 
1  0 


0  1 

a  b 

c  d 

1 - 

O 

c  d 

a  b 

,  compute 


Example  III.  20  Given  A  = 
Solution: 

EA  = 


Notice  that  E  interchanges  the  two  rows  of  A. 

2.  Scalar  Multiplication  of  a  Row 

This  operation  replaces  a  row  of  a  matrix  with  a  scalar  multiple  of  the  row. 


Example  III.  21  Given  A  = 


pute  EA. 


"  1 

2 

3 

4  ‘ 

’  1 

0 

0 

0  ‘ 

5 

6 

7 

8 

and  E  = 

0 

-2 

0 

0 

9 

10 

11 

12 

0 

0 

1 

0 

.  13 

14 

15 

16  . 

.  0 

0 

0 

1 . 

com- 
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Solution: 


1 

0 

0 

0 

1 

2 

3 

4 

1 

2 

3 

4 

0 

-2 

0 

0 

5 

6 

7 

8 

-10 

-12 

-14 

-16 

0 

0 

1 

0 

9 

10 

11 

12 

9 

10 

11 

12 

0 

0 

0 

1 

13 

14 

15 

16 

13 

14 

15 

16 

Notice  that  E  multiplies  the  second  row  of  A  by  —2. 


Example  III.22  Given  A  = 


3  5 

-2  7 


and  E  = 


4  0 
0  1 


compute  EA. 


Solution: 


EA  = 


i 

o 

1 

3  5 

12  20 

1 - 

o 

1 — ‘ 

1 _ 

i 

r— 

csi 

i 

_ i 

-2  7 

Notice  that  E  multiples  the  first  row  of  A  by  4. 


3.  Row  Replacement 

This  operation  will  add  a  scalar  multiple  of  one  row  to  another  row.  Again  we 
begin  with  an  identity  matrix. 


f  1  0  0 


Example  III. 23  Given  I  = 


0  1  0 
0  0  1 


twice  the  third  row  to  the  first  row  of  I. 


form  an  elementary  matrix  which  adds 


Solution: 


1  0  2 


E  = 


0  1  0 


0  0  1 


"12  3' 

"  1  0  2' 

Example  III. 24  Given  A  = 

4  5  6 

7  8  9 

and  E  = 

0  1  0 

0  0  1  _ 

compute  EA. 
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Solution: 


1  0  2 

1  2  3 

15  18  21 

0  1  0 

4  5  6 

=: 

4  5  6 

0  0  1 

7  8  9 

7  8  9 

Notice  that  E  adds  twice  the  third  row  to  the  first  row  of  A. 

In  each  of  the  previous  examples,  the  elementary  matrices  performed  one  and 
only  one  elementary  operation.  However,  we  can  multiply  many  elementary  matrices 
to  form  a  single  matrix,  A,  that  will  perform  those  same  elementary  row  operations 
at  one  time. 


Example  III. 25  Given  A  = 


pute  EA. 


Solution: 


EA  = 


0  0  0  1 

0-200 

-1  0  10 

1  0  0  0 


’  1 

2 

3 

4  ' 

'  0 

0 

0 

1 ' 

5 

6 

7 

8 

and  E  = 

0 

-2 

0 

0 

9 

10 

11 

12 

-1 

0 

1 

0 

.  13 

14 

15 

16  . 

1 

0 

0 

0  . 

1 

2 

3 

4 

13 

14 

15 

16 

5 

6 

7 

8 

-10 

-12 

-14 

-16 

9 

10 

11 

12 

8 

8 

8 

8 

13 

14 

15 

16 

1 

2 

3 

4 

com- 


In  this  example,  E  actually  performs  three  elementary  row  operations. 


F.  EXERCISES 


1.  Given  T  :  II2 

tion? 


H3  and  T 


VL 


®i 

X2 


X\  —  x2 

0 

(x-i  +x  2f 


is  T  a  linear  transforma- 


61 


2.  Given  T  :  ft2  — *  K3  and  T 

tion? 


Xi 

\ 

.  x 2  - 

/ 

3a:i 

2xi  —  X2 
*2 


,  is  T  a  linear  transforma- 


3.  Given  T  :  %2  — *  K3  and  T 

tion? 


*x 

X2 


Xi 

Xi  +  4X2 
X1X2 


,  is  T  a  linear  transforma- 


4.  Given  the  following  matrices,  compute  the  1-norms. 


1  -3  2 

0  3  4 

1  -2  4 

7  3-2 

b)  B  = 

8  9  1 

c)  C  = 

2  4  0 

5  8  0 

6  0  0 

-1  7  -2 

5.  Given  the  following  matrices,  compute  the  oo-norms. 


1  -3  2 

0  3  4 

1  -2  4 

7  3-2 

b)  B  = 

8  9  1 

c)  C  = 

2  4  0 

5  8  0 

6  0  0 

-1  7  -2 

6.  Given  the  following  matrices,  determine  whether  they  are  elementary  matrices 
or  not.  If  they  are  elementary  matrices,  state  which  operation  they  perform  on  a 
compatibly-sized  matrix. 


1  0 

0  -4 


b)  E2  = 


1  0 
1  0 


c)  E3  = 


1  0  0 
0  0  1 
0  1  0 
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a)  Ei  — 


d)  e4  = 


1  0  -3 

1  0  0 

-2  0  0 

0  1  0 

e)  E5  = 

-1  0  1 

f)  e6  = 

0  1  0 

0  0  1 

0  0  1 

0  0  1 

0  0  1 

-1  1  0 

1  0  0 

7.  Given  Ei  = 

0  1  0 

,e2  = 

0  1  0 

, and  E3  = 

0-2  0 

1  0  0 

0  0  1 

0  0  1 

a)  What  elementary  row  operations  do  the  elementary  matrices  Ei,E2,  and  E3 
perform? 

b)  Compute  E1E2E3. 

c)  Compute  E3E2E1 . 

d)  Are  the  solutions  to  parts  b  and  c  equivalent?  Why  or  why  not? 


8.  Given  A  = 


-3  2  1  0 

0  10-4 

0  0  2  6 

0  0  0  5 

0  0  0  0 


a)  Are  the  columns  linearly  independent,  or  linearly  dependent? 

b)  Are  the  rows  linearly  independent,  or  linearly  dependent? 

c)  How  many  linearly  independent  columns  in  A? 

d)  How  many  linearly  independent  rows  in  A? 

e)  What  is  the  rank(A)? 
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9.  Given  A  = 


-1  2  0  4  5  -3 

3-72  0  1  4 

2-52  4  6  1 

4  -9  2  -4  -4  7  _ 

a)  What  is  the  rank(A)? 

b)  Compute  the  solution  set  x  to  the  equation  Ax  =  0. 
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IV.  SYSTEMS  OF  LINEAR  EQUATIONS 
AND  SOLVING  LINEAR  SYSTEMS 


The  first  step  in  solving  any  real-world  problem  mathematically  is  defining 
the  variables  and  the  relationships  between  the  variables.  The  variables  might  be, 
for  example,  time,  distance,  speed,  costs,  materials  or  profit.  The  relationships  might 
be  known  formulas  such  as  velocity  =  ^^ce.  Once  we  have  defined  the  variables 
and  we  have  formed  equations  using  the  relationships  which  describe  the  problem, 
we  can  turn  to  mathematics  to  solve  the  problem.  The  problems  we  will  be  dealing 
with  in  this  text  are  linear  systems.  There  are  many  ways  to  solve  systems  of  linear 
equations.  One  method  for  solving  them  is  the  method  of  substitution.  This  is  the 
most  fundamental  of  all  the  methods.  It  requires  that  we  solve  for  a  single  variable 
in  one  equation  and  then  substitute  the  result  into  the  other  equations.  This  is  a 
standard  method  for  two  or  three  variables,  but  for  real-world  problems  with  more 
then  three  variables  substitution  is,  at  best,  cumbersome.  In  this  we  will  look  at  three 
methods  for  solving  linear  systems:  Gaussian  elimination,  matrix  inversion,  and  LU 
decomposition.  Each  has  its  advantages  and  disadvantages.  Additionally,  there  are 
several  ways  of  finding  the  LU  decomposition  of  a  matrix.  We  will  discuss  a  “cooke- 
book”  form  of  LU  decomposition,  and  another  form  which  takes  advantage  of  the 
properties  of  matrix  block  multiplication.  This  second  form  is,  initially,  slightly  more 
difficult.  However,  it  is  actually  a  simpler  form  when  using  computers  to  perform  LU 
decomposition  on  large  systems.  We  will  look  at  the  errors  and  costs  of  the  three 
methods,  in  terms  of  the  number  of  computer  computations  required  and  the  time 
required  to  perform  the  calculation. 

A.  SYSTEMS  OF  LINEAR  EQUATIONS 

A  linear  equation  is  an  equation  which  can  be  written  in  the  form 
aix  i  +  a2x  2  +  •  •  •  +  anxn  =  b, 
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where  the  coefficients  a,  and  b  are  real  constants  and  the  x,  are  variables,  or  unknowns, 
r  i  t 

The  vector  x  =  x\  ‘  xn  is  a  solution  if  and  only  if  it  satisfies  the  linear 

equation. 

Definition  IV.  1  An  equation  of  the  form  a\X\  +  a2x2  +  •  •  •  +  anxn  =  b  is  linear  if 
the  following  properties  hold: 

1.  Every  variable  occurs  only  in  the  first  power. 

2.  There  are  no  products  of  variables. 

3.  No  variables  are  arguments  for  radical,  exponential,  logarithmic  or  trigono¬ 
metric  functions. 

Example  IV.  1  Given  the  following  equations,  determine  whether  they  are  linear  or 
nonlinear.  If  nonlinear,  state  why. 

a)  xix2  +  3x2  =  1 

b)  2xi  +  Sx2  =  sin  tv 

c)  sin  xi  +  x2  =  3 

d)  y/Zx-t  +  |x2  =  0 

e)  xi  +  (x2)3  =  5 
Solution: 

a)  Nonlinear,  because  XiX2  is  a  product  of  two  variables. 

b)  Linear.  Remember  sin  w  is  a  constant. 

c)  Nonlinear,  because  X\  is  the  argument  of  trigonometric  function. 

d)  Linear. 

e)  Nonlinear,  because  x2  occurs  to  a  power  higher  than  1. 

The  linear  equation  3xj  —  x2  =  6,  represents  a  line  in  V?  and  has  as  one 
T  -iT  r  1T 

solution  x  =  xi  x2  =20  •  This  solution  is  not  unique.  In  other  words, 

there  are  other  values  for  Xi  and  x2  which  satisfy  the  linear  equation.  Can  you  find 

r  1 T 

them?  Try  x  =  3  3 
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We  can  form  a  system  of  linear  equations,  which  might  describe  a  real  world 
problem,  using  several  linear  equations. 


al,lxl + 

Ol,2®2  +  ' 

m^n  —  ^1 

a2,la:l  + 

02,2^2  +  ' 

&2m%n  —  b2 

On 1®1+ 

an  2X2+  • 

tinman  —  bn 

By  making  a  slight  modification,  the  above  system  becomes  the  vector  equation, 


«l,l 

Ol,2 

^lm 

’  h  * 

02,1 

«2,2 

&2m 

b2 

X1 

• 

+  x2 

- 

H - b  xn 

• 

• 

Onl 

On  2 

O'nm 

bn 

When  a  system  of  linear  equations  is  written  in  the  vector  form,  we  can  more  easily 
discuss  the  geometric  interpretation  of  adding  and  subtracting  vectors.  This  form 
is  also  helpful  when  discussing  linear  combinations  of  vectors.  The  matrix  equation 
Ax  =  b  is  another  way  of  representing  a  system  of  linear  equations  and  has  its  own 
desirable  qualities. 

If  we  make  another  slight  modification  to  the  system  of  linear  equations  we 
get  the  matrix  equation, 


Ol,l  Ox, 2  • 

^lm 

X\ 

bi 

02,1  02,2  ’ 

*  a2m 

X2 

— 

b2 

Onl  On2  •  • 

Q"nm 

% n 

1 _ 

This  form  allows  us  to  perform  matrix  algebra  on  the  system  of  linear  equations  in 
order  to  determine  the  solution.  A  disadvantage  of  this  form  as  opposed  to  the  vector 
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form  is  that  our  geometric  interpretation  of  vectors  and  the  idea  of  linear  combinations 
of  vectors  seems  lost.  If  we  remember  to  think  of  columns  as  vectors,  then  our 
interpretations  and  ideas  concerning  vectors  will  also  hold  true  for  the  columns  of 
A  in  the  matrix  equation.  Taking  this  a  step  further,  our  interpretations  and  ideas 
concerning  vectors  can  be  extended  to  the  system  of  linear  equations.  We  can  take 
this  step  since  the  vector  equation,  matrix  equation  and  the  system  of  linear  equations 
are  really  different  ways  of  representing  the  same  thing.  The  different  forms  help  us 
discuss  different  ideas. 

Definition  IV. 2  Given  the  system  of  linear  equations  of  the  form, 

al,lxl~^  al,2x2~i~  ‘  ‘  '+  ®1  mxn  =  ^1 

02,1^1  “f"  0*2,2X2~\'  '  '  '  Q‘2mxn  ~  ^2 

aJila:l+  0>n2x2Jr  '  ‘  •+  anmxn  = 

the  matrix  of  coefficients  is 

al,l  °1,2  '  ■  *  alm 

a2,l  a2,2  '  ■  ■  a2m 

&nl  ^n2  *  *  *  ^ nm 

and  the  augmented  matrix  is 

#1,1  ^1,2  *  *  *  alm  | 

u 2,1  a2,2  *  *  *  a2m  | 

a ri\  U n2  *  ^ nm  \ 
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Notice  that  the  coefficients  of  the  system  of  linear  equations  form  the  matrix  of 
coefficients.  When  we  append  the  constants  from  the  right-hand  side  of  the  system 
of  linear  equations  to  the  matrix  of  coefficients,  we  get  the  augmented  matrix. 

Before  we  demonstrate  how  to  solve  systems  of  linear  equations,  we  must  first 
define  some  terms  which  describe  our  system. 

Definition  IV. 3  If  the  system  has  more  variables  than  equations  (m  >  n),  then 
the  system  is  said  to  be  underdetermined.  If  the  system  has  fewer  variables  than 
equations  (m  <  n),  then  the  system  is  said  to  be  overdetermined. 

Example  IV. 2  Given  the  following  system  of  linear  equations,  determine  whether 
the  system  is  underdetermined  or  overdetermined. 

*1  +  x2  +  *3  =  3 

2*i  -|-  2*2  T  2*3  —  1 

Solution:  The  system  is  underdetermined,  since  there  are  more  variables  than  equa¬ 
tions. 

Example  TV. 3  Given  the  following  system  of  linear  equations,  determine  whether 
the  system  is  underdetermined  or  overdetermined. 

*i  +  *2  =  1 

2*i  +  2*2  =  2 

3*i  +  3*2  =  3 

4*i  +  4*2  =  4 

5*i  +  5*2  =  5 

6*i  +  6*2  =  6 

Solution:  The  system  is  overdetermined,  since  there  are  more  equations  than  vari¬ 
ables.  Regardless  of  whether  a  system  is  overdetermined  or  underdetermined,  a  solu¬ 
tion  is  not  guaranteed. 

Definition  IV.4  If  a  solution  to  a  system  of  linear  equations  exists,  then  the  system 
is  consistent.  If  no  solutions  exist,  then  the  system  is  inconsistent. 

Example  IV.4  Given  0*i  +  0*2  =  2,  solve  for  x. 
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Solution:  The  equation  is  inconsistent.  Whatever  values  are  chosen  for  the  variables, 
the  left-hand  side  of  the  equality  is  always  0  while  the  right-hand  side  is  2. 

We  have  shown  what  a  linear  system  is  and  the  different  ways  to  represent  it. 
We  have  also  shown  that  a  system  may  or  may  not  have  a  solution.  Now  let’s  move 
on  to  finding  a  solution,  if  it  exists.  First,  we  start  with  the  method  of  substitution. 
This  method  is  perhaps  the  most  labor-intensive  of  all  the  methods  we  will  show. 

Example  IV. 5  Given  the  following  system  of  linear  equations,  determine  a  solution 
by  the  method  of  substitution. 

2x\  +  3x2  =  6 
X\  +  X2  —  2 

Solution:  Solve  for  X\  in  the  first  equation: 

2xi  +  3x2  =  6 

2xi  =  6  —  3^2 

„  3 

xi  =  3  -  -x2. 

Substitute  X\  =  3  —  |x2  into  the  second  equation: 

X\  +  X2  =  2 
3 

3  —  —X2  +  X2  =  2 

~h  =  -1 


X2 


2. 


Substitute  X2  =  1  into  either  of  the  original  equations: 

Xi  -(-2  —  2 

x\  =  0. 

r  1T 

The  solution  to  the  system  is  x  =  0  2 

As  the  numbers  of  equations  and  variables  increase,  the  method  of  substitution  can 
become  overwhelming.  Imagine  writing  aqo  in  terms  of  x9,  x$,  x7,  x6,  x5,  x4,  x3,  x2,  and 
X\.  This  is  obviously  not  the  most  efficient  method. 
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B.  GAUSSIAN  ELIMINATION 

Gaussian  elimination  is  difficult  to  explain  in  plain  English,  so  we  will  use 
examples  to  demonstrate  the  process.  First  lets  look  at  a  system  of  linear  equations 
and  try  eliminating  variables. 

Example  IV. 6  Given  the  following  system  of  linear  equations,  compute  the  solution 
x. 


2xi 

+ 

4x2 

-  2x3  = 

-4 

3x! 

+ 

X2 

-  2x3  = 

-5 

9xi 

+ 

6x2 

-  6x3  = 

3 

Solution:  Start  by  dividing  the  first  equation  by  2  and  the  last  equation  by  3.  This 
gives 

X\  +  2x2  —  lx3  =  —2 

3xi  +  x2  —  2x3  =  —5 

3xi  +  2x2  —  2x3  =  1. 

Eliminate  Xi  by  multiplying  the  first  equation  by  —3  and  adding  it  to  the  other 
equations.  This  gives 

lxj  +  2x2  —  x3  =  — 2 

-  5x2  +  x3  =  1 

-  4x2  +  x3  =  7. 

Eliminate  x2  by  multiplying  the  second  equation  by  —  |  and  adding  it  to  the  last 
equation,  which  gives  us  a  system  easily  solved: 

lxi  H-  2x2  —  x3  =  — 2 

-  5x2  +  #3  =  1 

+  1*3  =  f. 

We  can  perform  this  process  of  eliminating  variables  on  an  augmented  matrix 
using  elementary  row  operations.  This  process  is  known  as  Gaussian  elimination. 
The  following  are  elementary  row  operations. 

a)  Scaling:  Multiplication  of  a  row  by  a  scalar. 

b)  Row  interchange:  Interchange  of  two  rows. 
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c)  Row  replacement:  Addition  of  a  scalar  multiple  of  one  row  to  another  row. 
The  previous  system  of  linear  equations  can  be  written  as 

1  2  -1  |  -2 

0-5  1  |  1  , 

Jo  0  I  I  f  _ 

and  is  said  to  be  in  row  echelon  form.  An  augmented  matrix  is  in  row  echelon  form 
when  all  of  the  following  are  true: 

a)  Any  row  or  rows  consisting  entirely  of  zeros  occurs  as  the  last  row  or  rows. 

b)  The  leading  non-zero  entry  of  any  row,  known  as  the  pivot,  is  to  the  left  of 
all  pivots  in  subsequent  rows. 

c)  All  entries  below  pivots  are  zero. 


Example  IV. 7  Given  A  = 
reduce  it  to  row  echelon  form. 


6  4-2 
3  1  -2 
9  6  3 


-4 

-5 

6 


use  elementary  row  operations  to 


Solution:  Replace  row2  by  — |rowl  +  row2  and  replace  row3  by  — |rowl  +  row3. 
This  gives 

6  4-2|-4 

0-l-l|-3  • 

0  0  6  |  12 

The  augmented  matrix  is  now  in  row  echelon  form  and  is  itself  a  system  of  linear 
equations  that  can  be  expressed  as 

6#!  +  4x2  —  2x3  =  —4 
—  x2  —  x3  =  —3 

6x3  =  12. 


Definition  IV.5  If  two  augmented  matrices  have  the  same  solution  set,  then  they 
are  said  to  be  equivalent. 
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Theorem  IV.  1  If  any  elementary  row  operation  is  performed  on  a  augmented  matrix 
A  (E  ,  then  the  resulting  augmented  matrix  has  the  same  solution  set  as  the 

original  augmented  matrix. 


Using  this  theorem,  we  can  verify  that  our  row  echelon  form  matrix  from  the  previous 
example  is  equivalent  to  the  original  matrix.  In  order  to  determine  the  solution  set, 
we  will  solve  for  the  variable  in  the  last  row  and  then  substitute  it  into  the  row  above 
to  solve  for  the  next  variable  and  repeat.  This  process  is  called  back-substitution  for 
obvious  reasons. 


Example  TV. 8  Given  A  = 


6  4-2|-4 

0  -1  -1  j  -3 
0  0  6  |  12 


back-substitution  to  compute  the  solution  set. 


from  the  previous  example,  use 


Solution:  Writing  this  augmented  matrix  as  a  system  of  linear  equations  yields 

6xi  +  4x2  —  2x3  =  —4 

—  x2  —  x3  —  —3 

6^3  =  12. 

Starting  with  the  last  equation,  solve  for  x3,  then  use  repeated  back-substitution  to 

solve  for  x2  and  aq.  Therefore,  from  the  equation  Qx3  =  12,  we  have  x3  —  2. 

Back  substitute  x3  =  2  into  the  second  equation: 


— x2  —  x3  =  —3 
—x2  —  2  =  -3 
— x2  =  —3  +  2 
x2  =  1. 


Back  substitute  x2  =  1  and  x3  =  2  into  the  first  equation: 


6aq  +  4x2  —  2x3  =  —4 
6®i  +4(1)  -2(2)  =  -4 


6^! 


-4-4  +  4 
_2 
_ 3 
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1 1 

Therefore,  the  solution  is  x  =  -|  1  2  •  Notice  that  this  solution  is  unique. 

Also,  since  a  solution  does  exist,  the  system  of  linear  equations  is  consistent.  If  the 
solution  is  correct,  each  equation  should  be  satisfied  simultaneously  as  follows: 

6xi  +  4x2  —  2x3  =  6(— |)  +  4(1)  —  2(2)  =  —4 

3x\  +  x2  —  2x3  =  3(— |)  +  1  —  2(2)  =  —5 

9xi  T  6x2  +  3x3  =  9(— §)  +  6(1)  +  3(2)  =  6. 

Therefore,  x  is  the  solution  to  the  original  system. 

T  i  —2  I  3 1 

Example  IV.9  Given  A  =  1  _  i  2  _  3;  |  0  '  US6  Gaussian  elimination  to  re* 
duce  it  to  row  echelon  form. 

Solution:  Multiply  rowl  by  —  i  and  row2  by  1  +  i.  This  makes  the  elements  in  the 
first  column  of  A  real.  The  result  is 

1  2  i  |  —3* 

_  2  5-i  |  0  _ 

Using  Gaussian  elimination,  replace  row2  with  — 2rowl  +  row2,  which  yields  the  row 
echelon  form 

1  2  i  |  -3  i 

0  5-5*  |  6 i 


Example  IV.  10  Given  A  = 
pute  the  solution  set. 


1  2  i  |  -3  i 

0  5  —  5i  I  6 i 


use  back-substitution  to  com- 


Solution:  Multiply  row2  by  5  T  5 i  to  make  the  element  in  the  second  column,  second 
row  real.  This  gives 

1  2  i  I  —3  i 

A  = 

0  50  |  —30  +  30i 

Therefore,  from  the  second  row,  we  have  50x2  =  —30  +  30i,  which  yields  x2  =  ~3^~3t  ■ 
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Back  substitute  X2  —  into  the  second  equation: 


x-i  +  2  i 


X\  +  2ix2  =  —3  i 
— 3  + 

-T-)  =  -3’ 


X\  = 
i  T 


6  —  9  i 
5  ' 


Therefore,  the  solution  is  x  =  ~g±3i  J  . 

By  applying  a  few  more  elementary  row  operations  to  an  augmented  matrix, 
we  can  compute  the  solution  directly  instead  of  using  back-substitution. 


'  6 

4 

-2  | 

-4  ' 

0 

-1 

-1  1 

-3 

0 

0 

6  1 

12  _ 

Solution:  Divide  row3  by  6  to  obtain 


6 

4 

-2  | 

1 - 

1 

0 

-1 

-1  1 

-3 

1 - 

O 

0 

1  1 

- 1 

CM 

Replace  row2  by  row3  +  row2  and  replace  rowl  by  2row3  +  rowl;  this  yields 

6  4  0  |  0 

0  -1  0  |  -1  • 

0  0  1  |  2 

Replace  rowl  by  4row2  +  rowl  then  replace  row2  by  — row2  to  get 

600|-4 
0  1  0  |  1  • 

0  0  11  2 


Replace  rowl  by  |rowl  to  get 


1  0  0 
0  1  0 
0  0  1 
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Solving  for  the  variables  directly  yields  x\  =  — |,£2  =  1,  and  —  2,  or  x  — 

IT 

2  1  2 


-2  1 
3 


.  The  elimination  process  that  we  performed  is  called  Gauss- Jordan 
elimination.  It  puts  the  augmented  matrix  into  reduced  row  echelon  form  so  that  the 
solution  can  be  found  by  directly  solving  for  each  variable  vice  using  back-substitution. 
An  augmented  matrix  is  in  reduced  row  echelon  form  when  all  of  the  following  are 
true: 


a)  A  is  in  row-echelon  form. 

b)  All  pivots  are  1. 

c)  All  entries  above  the  pivots  are  zero. 

We  have  solved  systems  of  linear  equations  by  method  of  substitution,  and 
their  associated  augmented  matrices  by  Gaussian  and  Gauss-Jordan  elimination.  But 
are  these  solutions  valid  in  the  vector  and  matrix  equation  forms?  Lets  look  at  two 
examples  using  the  following  system  of  linear  equations,  that  shows  the  solution  set 
to  the  system  is  also  valid  for  the  equivalent  vector  and  matrix  equations. 


6xi 

+ 

4x2 

—  2  xz  = 

-4 

3xi 

+ 

X2 

—  2x3  = 

-5 

9x  i 

+ 

6x2 

+  3x3  = 

6 

As  shown  previously,  the  solution  to  this  system  is  x  = 


2 

3 


lT 


1  2 


'  6  ‘ 

"  4  ' 

‘  -2  ‘ 

'  -4  ' 

Example  IV.12  Given  the  vector  equation  X\ 

3 

9 

+X2 

1 

6 

+  #3 

-2 

3 

— 

-5 

6 

r  i  t 

determine  whether  x  =  —  |  12  is  a  solution. 


Solution:  Plugging  x  into  the  vector  equation  yields 


6 

4 

-2 

-4 

3 

+  1 

1 

+  2 

-2 

— 

-5 

9 

6 

3 

6 

We  see  that  x  is  a  solution. 
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Example  IV.  13  Given  the  matrix  equation 


mine  whether  x  = 


2 

3 


1  2 


'6  4  -2 ' 

’  £1  ' 

'  -4  ' 

3  1  -2 

= 

-5 

9  6  3 

.  X3  . 

6 

is  a  solution. 


deter- 


Solution:  Plugging  x  into  the  matrix  equation  yields 


6  4-2 

- 1 

1 

Wl  to 

_ 1 

1 

1 

_ 1 

3  1  -2 

1 

= 

-5 

9  6  3 

i 

to 

1 

1 _ 

We  see  that  x  is  a  solution. 

Now  that  we  can  solve  a  system  of  linear  equations,  we  define  some  terms 


which  apply  to  a  reduced  augmented  matrix. 


Definition  IV.6  Given  an  augmented  matrix  in  row  echelon  or  reduced  row  eche¬ 
lon  form,  the  variables  which  correspond  to  pivots  are  called  pivot  variables.  The 
remaining  variables  are  called  free  variables ,  and  can  take  on  real  parametric  values. 


Definition  IV.7  The  set  of  all  vectors  x  that  simultaneously  satisfies  all  equations 
in  a  system  of  linear  equations  is  said  to  be  the  solution  set  of  the  system. 


Example  TV.  14  Given  the  following  augmented  matrix  in  row  echelon  form,  deter¬ 
mine  the  pivot  variables,  free  variables  and  the  solution  set. 


'  1  3  -2  0  2  0  |  0  ' 
0  0  1  2  0  3  |  4 

0  0  0  0  0  1  j  | 
.00  0  0  0  0  j  0 . 


Solution:  The  variables  x\,x3,  and  xe  are  pivot  variables,  since  they  occupy  the 
leading  non-zero  entry  of  rows  1,2,  and  3.  The  remaining  variables  £2,  x4.  and  x5 
are  free  variables.  In  order  to  perform  back-substitution  on  the  augmented  matrix, 
we  must  first  set  the  free  variables  X2,X4,  and  £5  equal  to  some  parametric  values. 
We  will  use  r,  s,  and  t  €  7£,  respectively.  Then,  we  can  use  back-substitution  to 
solve  for  the  pivot  variables  in  terms  of  the  free  variables.  The  last  row,  because  it 
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consists  entirely  of  zeros,  adds  no  information.  Therefore,  starting  with  row  three, 
we  directly  solve  for  X6,  which  yields  x3  =  |.  Substitute  x&  —  |  into  row2,  and  make 
the  parametric  substitutions  X4  =  s  and  x5  =  t,  which  yields 

X3  +  2s  +  Ot  +  3(  — )  =  ^ 

x3  =  4  —  2s  —  1  =  3  —  2s. 


Substitute  x6  =  \,x5  =  t,x4  =  s,x3  =  3  -  2s,  and  x2  =  r  into  rowl  and  solve  for  x: 


X\  T  3r  2(3  -  2s)  +  Os  +  2t  +  0^ 


0 


The  solution  set  is  then  x  = 


x\  =  —  3r  +  6  —  4s  —  2t  =  6  —  3r  —  4s  —  2t. 

1 T 

6  —  3r  -  4s  —  2f  r  3  —  2s  s  t  |  ,  where  r,  s, 

and  t  €  71.  By  assigning  the  parameters  r,  s,  and  t  different  values,  we  can  find 
infinitely  many  solutions  to  the  system.  Note  that  because  a  solution  exists,  the 
system  is  consistent. 

Example  IV.  15  Given  the  following  augmented  matrix  in  row  echelon  form,  com¬ 
pute  the  solution  set. 

1  3  -2  2  |  0 " 

0  5  1  0  I  4 

0  0  -3  0  |  3 

0  0  0  6  j  2 

Solution:  xi,  x2-,  x3,  and  x4  are  pivot  variables.  This  system  has  no  free  variables. 

r  1 T 

Using  back-substitution,  we  find  that  x  =  ||  |  — 1  |  •  Because  a  solution 

exists,  the  system  is  consistent.  Also,  there  are  no  parametric  values,  therefore  the 
solution  is  unique. 


Example  IV.16  Given  the  following  augmented  matrix  in  row  echelon  form,  deter¬ 
mine  the  solution  set. 


■  1 

3 

-2 

0 

2 

0 

0  ' 

0 

0 

1 

2 

0 

3 

4 

0 

0 

0 

0 

0 

1 

1 

1  3 

.  0 

0 

0 

0 

0 

0 

1  2  . 
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Solution:  xi,X3,  and  x6  are  pivot  variables  and  X2,x4,  and  x5  are  free  variables. 
But,  in  the  last  row  we  have  0x4  +  0x2  +  OX3  +  0x4  +  Oxs  +  0x6  =  2,  which  means  that 
no  solution  exists,  and  therefore  the  system  is  inconsistent. 

In  previous  examples  we  saw  systems  with  no  free  variables,  which  led  to  a 
unique  solution.  We  also  saw  systems  with  free  variables,  which  led  to  either  an 
infinite  number  of  solutions  or  no  solutions. 

C.  EXISTENCE  AND  UNIQUENESS  OF  A  SOLUTION 

In  the  previous  section  we  reduced  the  augmented  matrices  into  their  row  eche¬ 
lon  or  reduced  row  echelon  form.  Then  we  solved  the  system  using  back-substitution. 
However,  a  solution  did  not  always  exist.  We  can  determine  whether  a  solution  exists 
or  not  by  looking  at  the  last  rows  of  a  row  echelon  matrix  or  a  reduced  row  echelon 
matrix.  If  a  solution  does  indeed  exist,  we  can  also  determine  whether  it  is  unique  or 
one  of  infinitely  many.  Recall  that  if  a  system  of  linear  equations  has  any  equations 
of  the  form 


Oxi  -)-  0x2  +  •  •  •  +  0xn  —  b ,  b  0, 


then  the  system  of  linear  equations  is  inconsistent.  However,  the  above  equation  may 
not  appear  in  the  augmented  matrix  until  after  Gaussian  elimination  is  complete  and 
the  augmented  matrix  is  in  row  echelon  or  reduced  row  echelon  form. 


1 

0 

0  ! 

_ 4  - 

6 

-2 

1 

0 

4 

3 

,  determine  the  solution  set 

1 

1 

0  1 

1 

3  - 

Example  IV.  17  Given  A  = 

Solution:  After  performing  Gaussian  elimination,  the  row  echelon  form  is 


1  0  0  |  -| 

0  1  0  I  1 

0  0  0  I  2 


79 


In  the  last  row  we  have  the  equation  Oxi  +  0x2  +  OX3  =  2.  No  matter  what  values 
we  choose  for  xl5x2,  and  x3,  the  equation  cannot  be  satisfied.  Therefore,  no  solution 
exists  to  the  system. 


7 

0 

-3  | 

-6  ' 

7 

9 

3 

3 

-7 

0 

0  | 

2 

Solution:  After  performing  Gaussian  elimination,  the  row  echelon  form  is 


- 1 

-a 

0 

-3  1 

1 

o> 

- 1 

0 

9 

0  1 

1 

co 

1 - 

0 

0 

3  1 

- 1 

OO 

This  augmented  matrix  does  not  contain  any  rows  of  the  form  Oxi  +  0x2  H - 1-  0xn  = 

b  b  ^  0.  Therefore,  a  solution  exists,  and  we  can  solve  for  each  pivot  variable  using 

r  -iT 

back-substitution.  Notice  that  the  solution  x  =  |  -|  |  is  unique  and  that 

the  number  of  pivot  variables  is  equal  to  the  number  of  equations. 


'9  9  -6  |  -2  ' 

Example  IV.  19  Given  A  =  0  3  —  6  j  5  ,  determine  the  solution  set. 

9  12  -i2  I  3 

Solution:  After  performing  Gaussian  elimination,  the  row  echelon  form  is 

96  0  |  -7 

0  3  -6  |  5 

0  0  0  |  0 

This  augmented  matrix  does  not  contain  any  rows  of  the  form  Oxi  +  0x2  H - f  0xn  = 

b,  b  ^  0.  Therefore,  a  solution  exists.  Notice  that  there  are  two  pivot  variables  and 
one  free  variable.  If  we  let  x3  be  the  parametric  value  t,  then  back  substitution  yields 

r  1r 

the  solution  set  x  =  _il± 1M  s±6t  t  ?  where  t  E  TZ.  Notice  that  there  are  fewer 

9  3 

pivot  variables  than  equations. 

Let’s  summarize  what  we  have  just  learned  from  these  examples.  Given  the 
row  echelon  or  reduced  row  echelon  form  of  the  augmented  matrix,  if  there  is  an 
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inconsistent  row,  then  the  system  has  no  solution.  If  the  number  of  pivot  variables 
is  equal  to  the  number  of  equations,  then  the  solution  is  unique.  If  there  are  free 
variables,  then  there  are  infinitely  many  solutions. 

If  the  right-hand  column  of  an  augmented  matrix  is  all  zeros,  then  the  system 
is  said  to  be  homogeneous.  If  the  right-hand  side  is  non-zero,  then  the  system  is  said 
to  be  non-homogeneous.  The  zero  vector,  or  trivial  solution ,  is  always  a  solution  to 
the  homogeneous  system.  A  non-trivial  solution  may  or  may  not  exist. 


Example  IV. 20  Given  the  following  homogeneous,  augmented  matrix  in  row  eche¬ 
lon  form,  determine  the  solution  set. 

‘  -1  3  -2  0  2  0  |0  ' 

0  0  -1  2  0  3  jo 

00  000  -1  |0 

A_  00  000  0  |0 

0  0  0  0  0  0  |0 

,  0  0  0  0  0  0  |0  . 


Solution:  As  stated  previously,  A  has  the  trivial  solution  x=|  0  0  0  0  0  0 
There  are  three  pivot  variables,  and  three  free  variables  which  means  we  can  find  a 
non-trivial  solution.  As  before,  we  set  the  free  variables  equal  to  parameters  r,  s,  and 
t  €  71.  Then  we  use  back-substitution  to  solve  for  the  pivot  variables,  which  yields 
x=l3r  —  4s  r  2 s  +  t  s  t  0 


D.  INVERSE  OF  A  MATRIX 

Repeated  Gaussian  elimination  on  a  large  matrix  is  inefficient  in  terms  of  the 
number  of  computer  computations.  Remember  that  our  augmented  matrix  comes 
from  some  real-world  problem  with  potentially  hundreds  or  thousands  of  variables. 
Therefore,  repeated  Gaussian  elimination  is  not  the  method  of  choice  for  solving 
systems  of  linear  equations.  Let’s  build  a  better  mousetrap  using  the  inverse  of  a 
matrix. 

So  far  we  have  looked  at  addition,  subtraction  and  multiplication  of  matrices. 
The  obvious  question  now,  is  what  about  division?  After  all,  we  have  developed  all 
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the  other  basic  algebraic  operations,  and  division  of  matrices  is  certainly  one  of  them. 
Unfortunately,  there  is  no  natural  way  to  talk  about  division  of  matrices.  To  get  a 
better  understanding,  we  need  to  go  back  to  some  very  basic  notions  about  algebraic 
operations,  and  how  these  basic  operations  work. 

Addition  and  subtraction  are  natural  concepts.  We  see  them  in  our  everyday 
life  whenever  we  count.  Multiplication  is  artificial,  it  is  really  just  a  quick  way  to 
perform  repeated  addition.  Division  is  also  artificial  and  comes  from  the  need  to 
solve  linear  equations  like  ax  =  b  given  that  you  know  the  scalars  a  and  b.  In  fact, 
you  do  not  have  to  know  general  division  to  solve  this  problem,  provided  you  know 
how  to  compute  a-1.  The  reciprocal  of  a,  also  known  as  the  multiplicative  inverse, 
enables  us  to  solve  linear  equations  using  multiplication;  x  —  a~lb. 

Example  IV. 21  Given  2x  =  6,  solve  for  x. 

Solution:  Multiply  both  sides  of  the  equation  by  the  multiplicative  inverse  which 
yields  x  =  3. 

Definition  IV.8  Let  A,B  G  1Znxn.  If  AB  =  BA  =  I,  we  say  that  B  is  the  multi¬ 
plicative  inverse  of  A,  denoted  B  =  A-1. 

Unfortunately,  there  are  some  difficulties.  First  of  all,  since  we  want  A-1A  = 
AA-1,  it  is  clear  that  only  square  matrices  can  have  inverses. 

Definition  IV. 9  A  matrix  is  nonsingular ,  or  invertible,  if  it  has  an  inverse,  and  is 
singular  if  it  does  not  have  an  inverse. 

At  this  point  in  the  text,  it  is  only  important  to  understand  what  an  inverse  is,  vice 
how  to  find  the  inverse  of  a  matrix.  Later  we  will  learn  how  to  find  the  inverse  of  a 
matrix  if  it  exists. 

We  can  solve  the  matrix  equation  Ax  =  b  by  left-multiplying  the  equation  by 

A'1  : 


82 


A-1  Ax  =  A-1b 
lx  =  A-1b 
x  =  A-1b. 

Now  we  need  to  find  A-1.  We’ve  already  shown  how  to  find  the  inverse  of  a  scalar. 
Since  only  square  matrices  have  inverses,  the  next  obvious  step  is  to  find  the  inverse 


of  a  2  x  2  matrix.  If  A  = 


A-x  = 


a  b 
c  d 


then  we  can  use  the  formula: 


1 


d  -b 

-c  a 


ad  —  be 

Notice  that  if  ad  —  be  =  0,  then  A-1  does  not  exist. 


,  provided  ad  —  be  ^  0. 


Example  IV.22  Given  A= 
Solution: 


2  -1 
5  3 


A"1  = 


This  is  easily  verified: 


6  +  5 


,  compute  A  1 . 


3 

1 

3 

1 

_ 

11 

11 

-5 

2 

5 

2 

11 

A  A 


AA"1  = 


_3  _1_ 

11  11 

5_  2_ 

'll  11 


2  -1 

5  3 


2  -1 

5  3 


A  J_ 

li  li 

_5_  _2_ 

'll  li 


1  0 
0  1 

1  0 
0  1 


=  I  and 


=  1. 


For  matrices  larger  than  2  x  2,  we  will  use  Gauss- Jordan  elimination.  Let’s  show 
why  this  will  work  using  elementary  row  replacement  matrices  to  reduce  an  invertible 
matrix  A  to  the  identity  matrix: 
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Efc*  •  -E3E2EiA  =  I 

(It  can  be  shown  that  this  is  possible  whenever  A-1exists.) 

Left  multiplying  the  matrix  equation  by  E^-  •  -E3E2Ei,  we  have 


E k'  ■  -E3E2EiAx  —  Ejf  •  -E3E2Eib 
lx  =  Efc-  •  -E3E2Eib 
x  =  Efc-  •  -E3E2Eib. 


Solving  Ax  =  b  using  the  inverse  gives  the  matrix  equation  x  =  A-1b.  Substituting 
x  =  Efc-  •  -E3E2Eib  into  the  matrix  equation  gives  Efc-  •  -E3E2Eib  =  A_1b.  Therefore, 
assuming  b  ^  0,  A-1  =  E*- • -E3E2Ei.  Elementary  matrices  are  cumbersome,  so 
instead  we  will  use  Gauss- Jordan  elimination  to  find  the  inverse  of  A.  This  is  done  as 
follows:  Place  the  identity  matrix  to  the  right  of  A,  to  form  a  new  matrix 


«1,1 

<*1,2  •  ' 

^lm 

1  i 

0  • 

••  0 

0-2,1 

^2,2  *  < 

&2m 

1  o 

1  •' 

••  0 

Onl 

2 

1  0 

0  -• 

'•  1 

Now  perform  Gauss-Jordan  elimination  on  the  entire  matrix.  The  product  of  the  ele¬ 
mentary  matrices  is  stored  in  the  right-hand  portion  of  the  new  matrix,  [I|Efc-  •  -E3E2Ei]  . 
By  the  preceeding  remarks,  the  matrix  on  the  right  is  A-1. 


Example  IV.23  Given  A  = 


4-2  6 

2  0-4 

2  0  2 


compute  A  1. 


Solution:  Form  a  new  matrix  using  A  and  I,  and  then  perform  Gauss-Jordan  elim¬ 
ination: 

4-2  6  |  1  0  0 

2  0  -4  |  0  1  0  ■ 

2  0  2  |  0  0  1 
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Replace  row2  by  —  |rowl  +  row2  and  replace  row3  by  — |rowl  +  row3,  which  gives 

1  0  0 


4-2  6  | 

0  1  -7  |  - 

0  1  -1  |  - 

Replace  row3  by  — row2  +  row3,  which  gives 


•1  i  o 
•I  0  1 


4  -2 
0 

0  0 


1 


1-71-1 


0  0 

1  0 

0  -1  1 


Replace  row3  by  |row3,  replace  row2  by  7row3  +  row2  and  replace  rowl  by  — 6row3 
+  rowl,  which  gives 


4-2  0 

0 

0  0  1 


1 


1  -1 


10|-1 


o  -l 


7 

6 

1 

6 


Replace  rowl  by  2row2  +  rowl  and  replace  rowl  by  Irowl,  which  gives 


1  0  0 
0  1  0 
0  0  1 


0-11 

w  a  cf 


The  inverse  matrix  is 


A"1  = 


l  i 
6  3 


1  =1 

2  6 

o  ^ 


Example  IV.24  Given  the  following  matrix,  compute  A  A 


A  = 


3  0-39 

2  4  0  -1 

0-164 
7  8-36 
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Solution:  In  order  to  simplify  some  of  the  calculations,  ensure  that  each  pivot  is  1. 


3 

0 

-3 

9 

1 

0 

0 

0 

1 

0 

-1 

3 

1 

3 

0 

0 

0 

2 

4 

0 

-1 

0 

1 

0 

0 

0 

4 

2 

-7 

—2 

3 

i 

0 

0 

0 

-1 

6 

4 

0 

0 

1 

0 

0 

-1 

6 

4 

0 

0 

1 

0 

7 

8 

-3 

6 

0 

0 

0 

1 

0 

- 

8 

4 

-15 

—7 

3 

0 

0 

1 

1 

0 

-1 

Q  I 

d  3 

0 

0 

0 

1 

0 

-1 

3 

1 

3 

0 

0 

0 

0 

1 

1 

2 

-7 

4 

-1 

6 

1 

4 

0 

0 

0 

1 

1 

2 

—7 

4 

-1 

6 

1 

4 

0 

0 

0 

0 

13 

2 

9 

4 

-1 

6 

1 

4 

1 

0 

0 

0 

1 

18 

52 

—2 

78 

2 

52 

2 

13 

0 

0 

0 

0 

-1 

-1 

—2 

0 

1 

0 

0 

0 

1 

1 

2 

0 

-1 

1 

0 

-1 

0 

-8 

3 

-6 

0 

-3 

0 

1 

1 

2 

0 

19 

12 

15 

4 

0 

-7 

4 

0 

0 

1 

0 

-58 

-34 

2 

18 

156 

52 

13 

52 

0 

0 

0 

1 

1 

2 

0 

-1 

1000=^ 


0  0  0  1  1 


l 

0 

-1 

0 

-8 

3 

-6 

0 

3 

0 

1 

0 

0 

276 

156 

212 

52 

-2 

26 

-100 

52 

0 

0 

1 

0 

-58 

—34 

2 

18 

156 

52 

13 

52 

0 

0 

0 

1 

1 

2 

0 

-1 

-346 

2 

174 

52 

13 

52 

212 

-2 

-100 

52 

26 

52 

-34 

2 

18 

52 

13 

52 

2 

0 

-1 

We  have  combined  a  few  of  the  steps  to  save  space.  During  the  elimination  process 
we  can  perform  row  replacements  and  scalar  multiplication  only.  Row  exchanges 
require  permutation  matrices.  In  order  to  maintain  simplicity  we  will  not  perform 
row  exchanges. 
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E.  TRIANGULAR  SYSTEMS 

Triangular  systems  have  a  very  useful  form.  We  have  already  shown  how  this 
form  can  be  utilized  to  solve  systems  of  linear  equations.  That  is,  every  time  we 
reduce  our  augmented  matrix  into  its  row  echelon  or  reduced  row  echelon  form,  we 
have  reduced  it  to  a  triangular  matrix.  In  the  next  section,  triangular  matrices  will 
be  used  often  when  performing  back-substitution  and  forward  elimination. 

Definition  TV.  10  In  an  upper  triangular  matrix,  all  of  the  entries  below  the  main 
diagonal  are  zero. 

Example  TV. 25  A  is  an  upper  triangular  matrix: 

"  2  —1  0  5  3  1  * 

0  73  08-6 

A_  0  0  0  -1  6  4 

.  0  0  0  0  3  2  . 

Definition  TV.  11  In  a  lower  triangular  matrix,  all  of  the  entries  above  the  main 
diagonal  are  zero. 

Example  IV.26  A  is  a  lower  triangular  matrix: 

'  1  0  0  0  |  0  ' 

5  2  0  0  |  0 

-3  0  -4  0  |  0  ‘ 

.  2  -5  9  0  |  0  . 

F.  LU  DECOMPOSITION 

LU  decomposition  is  another  method  used  to  solve  Ax  =  b.  In  real-world 
problems,  b  may  change.  If  Gaussian  elimination  is  used,  it  would  need  to  be  repeated 
for  each  new  b.  However,  LU  decomposition  allows  us  to  change  b  without  repeating 
Gaussian  elimination.  First  we  will  show  how  to  solve  LUx  =  b.  Then  we  will  discuss 
how  to  find  L  and  U. 

Let  A  =  LU,  where  L  is  a  lower  triangular  matrix  and  U  is  an  upper  triangular 
matrix.  Then  the  matrix  equation  becomes  LUx  =  b.  In  this  form  we  can  find 
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a  solution  by  performing  a  simple  forward  elimination  and  then  an  equally  simple 
back-substitution.  First  we  let  Ux  =  z,  which  gives 

Ax  =  LUx  =  Lz  =  b. 

Since  L  is  lower  triangular,  we  can  use  forward  elimination  to  solve  for  z,  and  then 
perform  back-substitution  on  Ux  =  z.  If  b  changes,  we  will  have  to  repeat  the  forward 
elimination  and  back-substitution  using  the  LU  we  have  already  found.  If  b  changes 
for  any  reason  in  the  augmented  matrix  we  will  have  to  repeat  Gaussian  elimination 
and  back-substitution.  After  a  few  homework  exercises  you  will  agree  that  forward 
elimination  is  less  costly  than  Gaussian  elimination  in  terms  of  computations  and 
time.  Now  all  we  have  to  do  is  find  a  L  and  a  U  which  satisfy  A  =  LU.  When  we 
performed  Gaussian  elimination  on  the  augmented  matrix,  the  result  was  an  upper 
triangular  system.  Then  we  used  back-substitution  to  solve  the  system.  Here,  we  will 
use  Gaussian  elimination  on  A,  instead  of  on  the  augmented  matrix. 

I"  2  —3  111 

Example  IV.27  Given  A  =  ^  Q  j  2  ,  use  both  Gaussian  elimination  and 

LU  decomposition  to  compute  the  solution  set. 

Solution:  Beginning  with  Gaussian  elimination,  replace  row2  by  —  |rowl  +  row2, 
which  gives 

2  -3  |  1 

0  -  I  - 

U  2  I  2  J 

Starting  with  the  last  equation,  solve  for  x2,  then  use  back-substitution  to  solve  for 
aq.  From  the  equation  |x2  =  f ,  we  get  x2  =  1.  We  now  substitute  x2  =  l  into 
the  first  equation: 

2xi  —  3(1)  =  1 

2xi  =  4 

Xi  = 
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2. 


1 1 

Therefore,  x  =  2  1 

Now  we  begin  the  LU  decomposition  by  performing  Gaussian  elimination  on  A  to  get 
an  upper  triangular  matrix: 

U  b  ~3 

[o  1 


We  now  need  a  lower  triangular  matrix  so  that  LU  =  A,  so  let  L  = 
easy  to  verify  that  LU  =  A.) 


Use  forward-elimination  on  the  augmented  matrix  to  solve  for  z  in  Lz  =  b, 


Starting  with  the  first  row,  solve  for  Xi,  then  use  forward-elimination  to  solve  for  x2 
in  the  second  row.  Therefore,  from  the  first  row,  Xl  =  1.  Forward-eliminate  xi  by 


substituting  xa  =  1  into  the  second  row: 


-(1)  +  x2 
X2 


2 

3 

2' 


Therefore,  x  =  1  §  J  * 

Use  back-substitution  on  the  augmented  matrix  to  solve  for  x  in  Ux  =  z, 


Starting  with  the  second  equation,  solve  for  x2,  then  use  back-substitution  to  solve 
for  X\.  Therefore,  from  the  equation  |x2  =  |,  we  have  x2  =  1.  Back-substitute 
x2  =  1  into  the  first  equation: 
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2®! -3(1)  =  1 


Therefore,  x  = 


1  |  .  Now  let’s  verify  that  the  L  we  chose  satisfies  LU  =  A. 


LU  = 


10  2-3 


5  10 


2  -3 
1  0 


'1641  [100' 

Example  TV.  28  Given  A  =  244  ,  b  =  |  1  2  —  1  ,  L  =  2  10  , 

— 1  2  5  J  -1  -1  1  _ 

'1  6  4  1 

and  U  =  0—8  —4  ,  compute  the  solution  set  to  Ax  =  b  using  the  LU  decom- 

0  0  5 

position  A  =  LU. 


Solution:  Solve  for  z  in  Lz  =  b,  which  gives  z  =  10  0 


Solve  for  x  in  Ux  =  z,  which  gives  x  =  1  0  0 

Verify  the  solution: 


1  6  4  1 


Ax  =  2  4  4 

-12  5 


2  =  b. 


Therefore,  our  solution  is  correct. 

We  get  U  for  free  by  performing  Gaussian  elimination  on  A.  But  we  still  need 
to  find  an  L.  Using  elementary  matrices,  we  can  find  a  method  for  building  the  matrix 
L.  We  use  elementary  matrices  to  perform  Gaussian  elimination  on  A  in  order  to  find 
a  U  =  E*-  •  -E3E2EiA.  We  then  left-multiply  Ax  =  b  by  E**  •  -E3E2Ei.  This  yields 

Efc-  •  -E3E2EiAx  =  Ejf  •  -E3E2Eib 


Ux  —  Efc-  •  -E3E2Eib. 
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Using  inverses,  we  solve  for  b: 


[Efc.  •  •E3E2E1]-1Ux  =  b. 

This  last  equation  should  look  familiar.  It  is  simply  LUx  =  b.  So  let  L  =  [E**  •  -E3E2Ei] 
If  we  use  the  same  principle  that  we  used  in  finding  the  inverse  of  a  matrix,  then  we 
can  find  the  product  of  E*-  •  -E3E2Ei.  But  remember,  we  will  perform  Gaussian  elim¬ 
ination  and  not  Gauss-Jordan  elimination. 


Example  IV.29  Given  A  = 


1  2  3 

-3  2  1 

2  -2  1 


compute  LU  such  that  A  =  LU. 


Solution:  Form  a  new  matrix  using  A  and  I  and  then  perform  Gaussian  elimination: 

1  2  3  |  1  0  0 

-3  2  1  |  0  1  0  • 

2  -2  1  |  0  0  1 

Replace  row2  by  3rowl  +  row2  and  replace  row3  by  — 2rowl  +  row3.  This  gives 

1  2  3  |  10  0 

0  8  10  |  3  10 

0-6-5|-201 

Replace  row3  by  |row2  +  row3.  This  gives 

12  3  |  1  0  0 

0  8  10  |  3  1  0  • 

0  0  -  I  1  -  1 

So,  we  have 


1  2  3 

1  0  0 

-1 

1  0  0 

u  = 

0  8  10 

,L  = 

3  1  0 

therefore  L  = 

-3  1  0 

_o  o  f_ 

I  2  j 

4  4 

2  -f  1_ 

91 


This  method  for  finding  L  should  look  familiar.  However,  don’t  mistake  it  for  the 
procedure  we  used  for  finding  the  inverse  of  a  matrix.  When  finding  L,  we  performed 
Gaussian  elimination.  Then  we  found  the  inverse  of  the  product  of  the  elementary 
matrices.  This  inverse  turned  out  to  be  L.  This  seems  like  more  work  than  is  necessary 
because  we  had  to  perform  Gaussian  elimination  and  then  had  to  find  the  inverse  of 
the  matrix.  There  is  a  simpler  method.  Take  a  close  look  at  L  and  at  the  scalar 
row  multipliers  used  in  the  Gaussian  elimination.  When  we  eliminated  the  a2,i  entry, 
we  used  the  scalar  3.  Notice  that  we  have  a  —3  in  the  l2,i  position  of  L.  When  we 
eliminated  the  03,1  entry,  we  used  the  scalar  —2.  Notice  that  we  have  a  2  in  the  l^i 
position  of  L.  When  we  eliminated  the  a3)2  entry,  we  used  the  scalar  |.  Notice  that 
we  have  a  -|  in  the  /3)2  position  of  L.  These  are  not  coincidences.  To  find  the  entries 
of  L,  we  take  the  negative  of  the  scalar  row  multipliers. 

In  the  following  example  we  will  use  the  simpler  method. 


Example  TV.30  Given  A  = 


1 

-3 

2 

1 


2  3 
2  1 
-2  1 
2  3 


1  ' 
0 
0 

4 


,  compute  LU  such  that 


A  =  LU. 


Solution:  Write  A  and  I.  These  will  become  L  and  U,  respectively: 


1  2  3 

-3  2  1 

2  -2  1 

1  2  3 


1 

0 

0 

4 


10  0  0 
0  10  0 
0  0  10 
0  0  0  1 


Perform  Gaussian  elimination  only  on  A  and  insert  the  negative  of  the  scalar  row 
multipliers  into  I. 

Eliminate  a2y.  replace  row2  by  3rowl  +  row2  and  insert  —3  into  /2,i- 
Eliminate  a3)1:  replace  row3  by  -2rowl  +  row3  and  insert  2  into  /3]i. 

Eliminate  a4ji:  replace  row4  by  -lrowl  +  row4  and  insert  1  into  I4)1. 

After  introducing  0’s  below  the  pivot  in  column  1,  we  have 


92 


1  2  3  1  1  0  0  0 

0  8  10  3  -3  1  0  0 

0  -6  -5  -2  2  0  1  0 

0  0  0  3  J  [  1  0  0  1 

Eliminate  replace  row3  by  |row2  +  row3  and  insert  —  |  into  since  a4) 2  and 

a4j3  require  no  elimination,  insert  zeros  into  ?4)2  and  /4i3,  obtaining 

12  3  1  1  [  •  1  0  0  0 

0  8  10  3  -3  1  0  0 

=  UL. 

0  0  I  i  2  -I  1  0 

0  0  0  3  J  [  1  0  0  1  _ 

Gaussian  elimination  is  now  complete.  Notice  that,  in  the  example  above,  we  did 
not  perform  Gaussian  elimination  on  the  identity  matrix.  Instead,  we  inserted  the 
negative  of  the  scalar  row  multiplier  into  the  identity  matrix.  If  no  scalar  row  mul¬ 
tiplier  was  required  to  eliminate  an  entry  in  A,  then  the  corresponding  entry  in  the 
identity  matrix  was  left  as  zero.  It  is  crucial  to  remember  that  we  do  not  perform 
row  exchanges. 

G.  BLOCK  LU 

The  previous  LU  decomposition  method  used  Gaussian  elimination  to  factor 
the  matrix  A  into  the  triangular  matrices  L  and  U,  and  works  well  when  solving  small 
systems  of  linear  equations  analytically.  But  what  happens  when  our  systems  become 
too  large  to  feasibly  solve  by  hand?  We  must  find  a  method  that  allows  computers 
to  do  the  work  for  us.  By  factoring  A  into  upper  and  unit  lower  triangular  matrices, 
we  can  develop  a  simple  way  to  solve  for  x  in  the  equation  Ax  =  b.  To  see  how  to 
construct  the  LU  factorization,  consider  the  following  block  partitioning  of  the  matrix 
equation  A  =  LU, 
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a  bT 

1 

0T‘ 

1 - 

a 

a  A 

ao-1 

L 

0  u 

where  L  and  U  are  lower  and  upper  triangular  matrices  with  smaller  dimensions  than 
the  original  A.  Provided  that  LU  =  A  -  ^abT,  we  have  the  factorization  we  are 
looking  for.  The  equation  above  is  called  a  Gauss  step,  and  shows  how  to  get  the 
first  row  and  column  of  the  LU  factorization  matrices.  Once  this  is  complete,  we 
only  need  to  compute  the  LU  factorization  of  the  Schur  complement  A  —  ^ab  .  This 
matrix  is  one  row  and  one  column  smaller  than  the  matrix  we  started  with.  Since 
it  is  clear  that  the  LU  factorization  of  a  1  x  1  matrix  A  =  [or]  is  [or]  =  [1]  [a],  we 
can  build  a  sequential  procedure  for  finding  the  factorization.  It  is  much  easier  to 
show  how  this  algorithm  works  by  showing  a  simple  example.  In  the  next  example, 
to  avoid  confusion,  we  will  use  subscripts  on  A,  L,  and  U. 


Example  IV.31  Given  A  = 


2  6  2 
-3  -8  0 
4  9  2 


,  compute  LU  such  that  LU  =  A. 


Solution:  Start  by  factoring  A  to  find  the  first  row  and  column  in  L0  and  Uo,  so  that 
A  =  L0Uo 

Therefore, 


o  bT 

i 

o 

iH 

1 _ 

o  bT 

1 - 

& 

> 

i-* 

ao  1  Li 

0  Uj 

o'  =  2,  a  = 


L0Uo 


-3 

4 


y= 


6  2 


— 


-8  0 

9  2 


,  and 


- , 

l»— 1 

lo 

lo 

2  6  2 

_ 3 

0 

_2 

i - 

Ito 

t-1 

i _ 

lo 

a 

i _ 

Compute  the  Schur  complement  A2  =  Ai  —  ^abT,  which  gives 
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A2  =  Ai  —  — abT  = 

1 

0 

00 

1 

1 _ 

1 

-3 

6  2 

a 

9  2 

2 

4 

O 

OO 

1 

l _ 

1 

CO 

1 

o> 

1 

1 _ 

1  3 

9  2 

12  4 

1 

1 

CO 

1 

to 

1 _ 

Factor  A2  to  find  the  second  row  and  column  in  Li  and  Ui,  so  that 


A2  — 


1  3 

-3  -2 

1  0T 
aaT1  Li 


,  a  =  1,  a  =  [—3] ,  bT  =  [3] ,  and  A3  =  [—2] ,  so 
1  0  0 


1 - 

3 

1 _ 

O 

_ 1 

§10 


2  -3  L2 

Compute  the  Schur  complement  A4  =  A3  —  -abT  so  that 


- 1 

to 

6 

to 

_ l 

0 

1 

3 

0 

_ 1 

0 

U2  _ 

A4  =  A3  -  -abT  =  [-2]  -  [-3]  [3]  =  [7] 
a 


Factor  A4  to  find  the  third  row  and  column  in  L2  and  U2,  which  gives 


A4 

A 


[7] ,  a  =  7,  a  =  [0] ,  br  =  [0] ,  and  A5  =  [0] ,  so 


a  bT 
a  A 


a 

0 


bT  ' 

1  0  0 

2  6  2 

u 

-f  1  0 

2-3  1 

0  1  3 

0  0  7 

=  LTJ. 


It  turns  out,  however,  that  for  a  general  matrix  A  €  Cnxn,  it  is  not  always 
possible  to  find  an  LU  factorization.  If  you  want  to  prove  this  to  yourself,  try  factoring 


A  = 


0  1 
1  1 
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Proceeding  in  the  normal  manner,  we  immediately  see  that  a  divide-by-zero  problem 
occurs  when  everything  below  the  first  pivot  is  divided  by  the  pivot.  Fortunately,  it  is 
always  possible  to  find  a  permutation  matrix  P  such  that  PA  has  an  LU  factorization. 
To  begin,  find  a  permutation  Pi,  and  partition  the  matrix 


PiA 


a  bT 

a  B 


Choose  Pi  such  that  either  a  ^  0,  or  both  a  =  0  and  a  =  0.  Since  the  general 
structure  of  L  and  U  is  known,  we  partition  them  as 


'  1  0T  ' 

T 

v  ir 

and  U  = 

v  L 

— i 

o 

Setting  PiA  =  LU  gives 


a  br 

V 

nT' 

a  B 

VV 

LU  +  vuT 

Equating  elements  and  simplifying  yields 


v  =  a 

1 

v  =  —a 
a 

uT  =  bT 

LU  =  B  —  — abT. 
a 


The  vector  uT  can  be  taken  directly  from  the  permuted  A,  and  both  L  and  U  come 
from  factoring  the  Schur  complement, 


B  -  -abT. 
a 


Of  course,  it  can  also  happen  that  we  must  apply  a  permutation  to  the  Schur  com¬ 
plement  to  get  a  non-zero  pivot.  That  is, 


or 


P2  LU  =  B  -  — abT, 
a 


in  which  case 


PiA 


’  1  0T 

a  bT 

-a  P^L 

a  * 

0  U 

But  this  violates  the  structure  of  L,  since  P2  L  is  not  unit  lower  triangular.  However, 
we  notice  that  if  we  let 


P2 


1  0T 
0  P2 


then 


P2P1A 


0T 

a  bT 

1  0T 

a.  bT 

Pit  _ 

0  U 

IP2a  L 

0  u 

This  is  a  valid  L  and  U.  The  reader  can  verify  that  the  process  can  be  continued  until 
we  have  a  complete  LU  factorization. 

In  practice,  it  is  usually  considered  best  to  apply  a  permutation  at  every  stage 
in  the  process  and  to  choose  it  so  that  the  pivot  is  as  large  as  possible.  In  effect,  we 
choose  the  permutation  that  puts  the  largest  element  of  the  first  column  into  the  first 
row.  This  strategy  is  called  partial  pivoting.  Another  strategy,  full  pivoting,  utilizes 
both  row  and  column  interchanges  to  put  the  largest  element  remaining  in  the  matrix 
into  the  first  row  and  column. 


H.  PIVOTING 

We  see  in  our  derivation  of  the  LU  factorization  that  pivoting  may  be  necessary 
to  prevent  division  by  zero.  However,  the  importance  of  pivoting  in  a  computer 
algorithm  goes  far  beyond  avoiding  the  divide  by  zero  errors.  As  mentioned  previously, 
pivoting  is  designed  to  get  the  largest  number  in  a  row  or  column  into  the  pivot 
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position.  Not  only  is  this  done  to  ensure  that  the  pivot  is  not  identically  zero,  but 
to  ensure  that  the  pivot  is  not  very  small.  Extremely  small  pivots  pose  problems  for 
two  reasons. 

The  first  reason  is  direct.  If  the  pivot  is  very  small,  meaning  much  less  (in 
magnitude)  than  1,  and  the  remaining  numbers  in  the  column  are  large,  division  by 
the  pivot  will  produce  very  large  numbers.  These  numbers  pose  a  potential  overflow 
problem  in  the  computer.  By  employing  partial  pivoting  to  choose  the  largest  element 
in  the  column  as  the  pivot,  this  catastrophe  is  eliminated  and  guarantees  that  all 
remaining  numbers  in  the  column  are  less  than  one.  Thus  we  can  prevent  an  overflow 
error  from  occurring. 

The  second  reason  for  pivoting  is  subtle,  but  far  more  important.  If  the  pivot 
is  small,  one  might  wonder  how  it  got  that  way.  When  performing  the  LU  decompo¬ 
sition,  if  the  first  pivot  of  A  is  small,  it  just  simply  occupies  the  first  pivot  position. 
However,  the  remaining  pivots  are  computed  to  be  small  due  to  the  repeated  subtrac¬ 
tion  in  the  Schur  complement.  Let’s  see  what  this  really  means.  Say  for  example  that 
we  have  two  numbers,  a  =  §  and  b  =  ^^5,  and  that  our  computer  is  a  rounding 
machine  that  uses  a  base  10,  4  digit,  floating  point  system.  The  computer  will  approx¬ 
imate  these  two  numbers  as  a  =  .3333  and  b  =  .3330.  The  rounding  errors  produced 
by  these  approximations  look  innocent  enough  since  the  approximations  are  so  close 
to  their  actual  values.  But  what  happens  when  the  machine  computes  a—  6?  Well, 
the  machine  represents  this  value  as  .3000  x  10~3,  which  is  quite  a  bit  different  from 
the  value  .3233  x  10-3,  the  value  closest  to  the  true  result  without  initial  rounding. 
The  phenomenon  just  shown,  known  as  catastrophic  cancellation ,  occurs  when  two 
very  small  numbers  that  are  nearly  equal  are  subtracted.  The  result  leads  to  a  great 
loss  in  precision.  One  way  to  lessen  the  effect  of  catastrophic  cancellation  is  to  avoid 
subtraction.  That  is  why  developers  of  computer  algorithms  spend  a  lot  of  time  find¬ 
ing  ways  around  the  use  of  subtraction.  However,  subtraction  is  not  the  underlying 
problem,  it  is  rounding  that  causes  the  problem  of  cancellation.  One  reasonable  way 
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to  avoid  cancellation  is  to  shy  away  from  small  numbers  that  were  calculated  from 
large  numbers  by  successive  subtractions.  Such  numbers  are  usually  filled  with  error. 
Therefore,  a  small  pivot  is  doubly  dangerous.  First,  because  it  is  probably  filled  with 
error,  and  second,  because  dividing  by  it  magnifies  the  error.  That  is  why  it  is  so 
important  to  pivot  when  developing  computer  algorithms.  This  applies  to  solving 
systems  of  linear  equations  using  both  LU  decomposition  and  Gaussian  elimination 
algorithms. 

I.  EXERCISES 


1 .  Determine  whether  the  following  equations  are  linear  or  nonlinear  in  the  variables 
x,  y  and  z.  If  nonlinear,  state  why. 

a)  x  +  y  +  2z  =  simr  b)  xy  -f  z  +  2  =  0  c)  x2  +  y  +  \/hz  =  1 

d )x  —  y  e)  6a:  — 3 y'1  =  z  f)  tan(7r)x  +  e2y  =  z  g)  sin(x)+y  —  5z  =  0 


2.  Given  the  following  systems  of  linear  equations,  use  the  method  of  substitution  to 
compute  the  solution  set. 

a:i  +  x2  =  1  3a;!  —  3x2  =  2 

a)  b) 

— 2a: i  +  3a:2  =  4  6a:i  •  +  x2  =  1 

2a:i  —  3a:2  —  xz  =  6 

c)  a;i  +  6a:2  —  2a:3  =  12 

—xi  +  4a:2  +  6x3  =  24 


3.  Given  the  following  system  of  linear  equations, 
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X\  -)-  2X2  "f"  ^3  —  3 

2xi  +  3x2  “t-  4x3  =  4 

4xi  +  5x2  +  9x3  =  5, 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  by  substituting  it  back  into  the  system  of  linear  equa¬ 
tions. 


4.  Given  the  following  system  of  linear  equations 


— 

2x2 

+ 

3x3  = 

1 

3xi 

+ 

6x2 

— 

3x3  = 

-2 

6xi 

+ 

6x2 

+ 

3x3  = 

5, 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  by  substituting  it  back  into  the  system  of  linear  equa¬ 
tions. 


5.  Given  the  following  system  of  linear  equations, 


X 

— 

y 

+  2  z 

—  w  = 

-1 

2x 

+ 

y 

-  2z 

—  2w  = 

-2 

— X 

+ 

2  y 

-  4z 

+  w  = 

1 

3x 

—  3  w  = 

-3, 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 
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c)  Verify  the  solution  by  substituting  it  back  into  the  system  of  linear  equa¬ 
tions. 

6.  Given  the  following  system  of  linear  equations 

—  x  +  2y  +  3>z  =  0 

w  +  x  +  4y  +  \z  =  7 

w  -f-  3x  -f-  7y  -(-  9z  =  4 

— w  —  2x  —  Ay  —  6z  =  6 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  set  by  substituting  it  back  into  the  system  of  linear 
equations. 


7.  Given  the  following  system  of  linear  equations 

Xi  +  x2  4-  2x3  =  5 

2xi  +  5x2  +  7x3  —  19 

2x\  +  4x2  +  6x3  =  16 

a)  Write  the  augmented  matrix. 

b)  Use  Gauss-Jordan  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  set  by  substituting  it  back  into  the  system  of  linear 
equations. 

8.  Given  the  following  system  of  linear  equations 

xi  +  2a:2  +  x3  =  6 

X\  +  2x2  +  2x3  =  7 

2xi  +  \x2  +  2x3  =  15 
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a)  Write  the  augmented  matrix. 

b)  Use  Gauss-Jordan  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  set  by  substituting  it  back  into  the  system  of  linear 
equations. 


9.  Given  the  following  system  of  linear  equations 

—X\  +  4x2  d”  xz  —  3 

X\  9x2  —  2x3  =  4 

6xa  +  4x2  —  8x3  =  5 

a)  Write  the  augmented  matrix. 

b)  Use  Gauss-Jordan  elimination  to  compute  the  solution  set. 

c)  Verify  the  solution  set  by  substituting  it  back  into  the  system  of  linear 
equations. 

10.  Given  the  following  system  of  linear  equations 

Xi  +  2x2  +  x3  =  i>i 

2xi  +  3x2  +  2x3  =  &2 

4xi  +  5x2  +  8x3  =  &3 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 

c)  Can  you  find  a  vector  b,  so  that  the  augmented  matrix  is  inconsistent? 

d)  Can  you  find  a  non-trivial  solution  if  b  =  0? 


1 1 .  Given  the  following  system  of  linear  equations 
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x-i  —  2x2  —  %3  =  h. 

—4xj  +  5x2  +  2xs  —  b2 

4xj  +  7x2  +  6x3  =  b3 

a)  Write  the  augmented  matrix. 

b)  Use  Gaussian  elimination  to  compute  the  solution  set. 

c)  Can  you  find  a  vector  b,  so  that  the  augmented  matrix  is  inconsistent? 

d)  Can  you  find  a  non-trivial  solution  if  b  =  0? 

12.  Given  the  following  system  of  linear  equations 

u  +  0  —  w  +  2x  +  y  +  z  =  —3 

—3  u  +u  +  0  —  6a;—  y  —  4z  =  11 

2u  +  5v  —  2w  +  4x  +  22y  —3  z  =  4 

—u  +  2v  +  0  +  8a:  +  27  y  +  27  z  =  —33 

a)  Determine  whether  the  system  is  consistent  or  inconsistent. 

b)  If  the  system  is  consistent,  compute  the  solution  set  by  any  method. 


13.  Given  A  = 


3  1 
5  2 


a)  Compute  A  1 ,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 


14.  Given  A  = 


3  6 
1  2 


a)  Compute  A  1 ,  if  it  exists. 
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b)  Is  A  singular,  or  nonsingular? 


15.  Given  A  = 


1  2  3 
1  1  2 
0  1  2 


a)  Compute  A  1 ,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 


16.  Given  A  = 


1  2  -1 
4  7-3, 
2  1  3 


a)  Compute  A  1,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 


17.  Given  A  = 


1  6  2 

-2  3  5, 

7  12  -4 


a)  Calculate  A  1 ,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 


18.  Given  A  = 


1-3  0-2 

3  -12  -2  -6 

-2  10  2  5 

-1  6  13 
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a)  Compute  A  *,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 


10  2  3 

-11  0  4 

19.  Given  A  = 

2  1-13 
-10  5  7 

a)  Compute  A-1,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 

20.  Given  A  = 

a)  Compute  A-1,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 

1  -i  0 

21.  Given  A  = 

0  1  +i 

a)  Compute  A-1,  if  it  exists. 

b)  Is  A  singular,  or  nonsingular? 

1  i  0 

22.  Given  A  =  —  i  0  1 

0  1  +  i  1  —  i 

a)  Compute  A-1,  if  it  exists. 
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b)  Is  A  singular,  or  nonsingular? 


3  0  0 

23.  Given  A  =  0  2  0,  compute  A-1. 

0  0  1 


4-2  7 

24.  Given  A  =  0  5  1  ,  compute  A-1 

0  0-1 


2  0  0 

25.  Given  A  =  —6  0  0  ,  show  that  A-1  does  not  exist. 

4  3  6 

26.  Given  upper  and  lower  triangular  matrices,  show  that  each  is  invertible  if,  and 
only  if,  its  diagonal  elements  are  nonzero. 

27.  Given  upper  triangular  matrices  A  and  B,  show  that  AB  is  also  upper  triangular. 


28.  Given  oi, 1,01,2,  02,1,  and  02,2  €  7^-,  such  that  01,102,2  01,202,1  7^  0,  compute 

Ol,l  Oi,2  &1,1  frl,2  1  0 

61,1,61,2,02,1,  and  02,2  such  that  — 

02,1  02,2  62,1  62,2  0  1 
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29.  Given  A  = 


,  compute  L  and  U  such  that  A  =  LU. 


3  -6  -3 
2  0  6 
-4  7  4 


30.  Given  A  = 


4  3  2  1 

8  9  6  3 

4  9  8  4 

8  9  10  6 


,  compute  L  and  U  such  that  A  =  LU. 


1  0  0 

1  2  3 

1 

31.  Given  L  = 

-3  1  0 

,u  = 

0  8  10 

,  and  b  = 

1 

,  compute  the 

2  -f  1 

4 

1 

1/5|<N 

O 

O 

1 

solution  set  to  LUx  =  b. 


5xj  +  5x2  +  10^3  =  0 

32.  Given  the  system  of  linear  equations  —8x1  —  7x2  —  9x3  =  1  ,  use  LU 

4x2  +  26x3  =  4 

decomposition  to  compute  the  solution  set. 

— Xi  —  3x2  —  4x3  =  —6 

33.  Given  the  system  of  linear  equations  Zx\  +  10x2  —  10x3  =  —3  ,  use 

— 2xj  —  4x2  +  llx3  =  9 

LU  decomposition  to  compute  the  solution  set. 
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2x2  +  ^3 


2 


4x 


34.  Given  the  system  of  linear  equations  —  12xi  +  3x2  =  5  ,  use  the 

8xi  —  X2  —  2x3  =  6 

Block  Partitioning  method  of  LU  decomposition  to  compute  the  solution  set. 
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V.  SOLUTIONS  TO  EXERCISES 

A.  SOLUTIONS  TO  CHAPTER  I 


la)  AT  = 


p  1 

lb)  B*  = 

1  —  Si  —6  1  +  i 

3  -6  1 

2  0  5 * 

lc)  C*  = 


2  +  6i  —  6  —  4i  1 

7  +  4 i  —  5  —  6i  3  +  i 
2-6-2  i  0 


2a)  AT  = 


5  +  i  7  — 2  % 


2b)  Bh  = 


1  +i  3 
-4  -7 i 


2c)  CH  = 


2  -6  14 

-4  i  -5  i 

22  -3  -  4i  10 


3a)  A  +  B  = 


8 

-7 

9 


3b)  B  and  D  are  not  compatibly-sized  for  addition. 


3  +  ‘hi  — 3  +  4z 


3c)  D-C  = 


—8  —  2i 


2-3  i 


addition. 


1  +  4z  4  —  6i 


3d)  CH  and  D  are  not  compatibly-sized  for 
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3e)  2A  —  3B  =  -9 

-22 


3f)  iDH  +  2CT  = 


8  +  3*  -4-12*  5  +  3* 

6-8 *  -3  +  2*  -6  +  9* 


4a)  A  —  B 


4b)  B  and  C  are  not  compatibly-sized  for  addition. 


4  —  3*  —5  —  4* 
4c)  D-^+C  =  —2  —  4*  —14* 

1  +  *  5 


4d)  CH-  D 


4  +  *  2  —1  —  * 

11+4*  —14*  5 


4e)  A  +  2B  = 


5  +  2* 
21 
26 


4f )  and  are  not  compatibly-sized  for  subtrac- 


3  6-2 


9  18  -8 


6  6  6 


6)  AB  =  6  12  -5  ~f~  -19  -20  -21  =  BA  Yes,  in  general  AB  ^  BA. 


15  18  21 


-1 

0 

0 

7-18* 

24  -  13* 

6 

4 

8 

_2  7b)  BtC  = 

— 9  +  2* 

7* 

-10+4* 

1 

-4 

-4 

-26  +  16* 

-26  +  26* 

-24  +  8* 
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7  +  18? 

-9-2* 

—26  —  16? 

—a 

0 

0 

7c)  CHB  = 

24  +  13? 

-7? 

-26  -  26? 

7d)  o(BA)  = 

4a 

8a 

-2a 

6 

1 

o 

T— 1 

1 

1 

to 

1 

oo 

a 

—4a 

—4a 

-1  -  14*  16  -  6*  2* 


7e)  BA  -I"  BC  — 


-4  +  8?  10  +  8?  -14  +  4? 

-5-2  i  —8  +  4?  -8  +  2?' 


8a)  AB  = 


15  33  41 
9  24  19 


8b)  BAt 


32  28 
16  14 
24  15 


8c)  CHB  = 


3  +  5?  9  +  16?  5  +  7? 

7-2*  14-4*  21  -  6* 


8d)  a(BC) 


a(  8  —  24?)  a(16  +  2  i) 
a(  4  —  12  i)  a(  8  +  i) 

—13  ai  a(17  +  i) 


8e)  AB  and  C  are  not  compatibly-sized  for  addition. 


B.  SOLUTIONS  TO  CHAPTER  II 


la)  xH  — 


-1-2*  2  +  3?  7 * 


lb)  yT  = 


-2  3 


lc)  x  +  y 


-3  +  2? 
5  —  3? 


1-7* 
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Id)  3x-2*y  = 


-3  +  10* 
6  —  15* 
-23* 


le)  x  and  z  are  not  compatibly-sized  for  addition. 


If)  x  +  z 


T  _ 


-1  +  * 

-2* 

-7  * 

lg)  2zH  = 

• 

00 

1 

1 

6-7* 

12 

lh)  z+z  = 


0  -4  12 


3a)  x  •  z  =  1  3b)  x  •  y  =  5 


-19 

-19 

3c)  x(z  •  y)  = 

-38 

3d)  (y  •  z)x  = 

-38 

-76 

-76 

-78 


4a)  x  •  z  =  39  4b)  (x  •  z)y  = 


117 

39 


-155 

-155 

4c)  x(z  •  y)  = 

62 

4d)  (y  •  z)x  = 

62 

31 

31 

6a)  32  +  8*  6b)  17 -17*  6c)  38  +  5*  6d)  32  -  8*  6e)  17+17* 

6f)  38  -  5 * 
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7a) 


-78 

-155 

-85 

-78 

117 

7b) 

62 

7c) 

119 

7d) 

117 

39 

31 

0 

39 

76  +  10* 

-102  -  68* 

-176  -  24* 

-76  + 10  i 

8a) 

109  -  53 * 

8b) 

85  -  17* 

8c) 

-32  -  128* 

8d) 

109  -  53 i 

53  +  109* 

-51  +  85i 

96  -  24* 

53  +  109*' 

9)  x  •  y  =  10,  not  orthogonal  x  •  z  =  0,  orthogonal  x  •  w  =  0,  orthogonal 
y  •  z  =  2,  not  orthogonal  y  •  w  =  0,  orthogonal  z  •  w  =  — 14,  not  orthogonal 


10a)  1 1 x | [  =  \J (4)2  +  (— 2)2  +  (2)2  =  V24  10b)  ||  y  ||  =  V62  10c)  ||z||  =  V30 

lOd)  x  and  y  are  not  orthogonal.  lOe)  z  and  y  are  not  orthogonal. 
lOf)  x  and  z  are  orthogonal. 


11a)  ||x||  =  \JxHx  =  \/59  lib)  ||y||  =  \/116  11c)  ||z||  =  \/23 

lid)  x  and  y  are  not  orthogonal.  lie)  z  and  y  are  orthogonal. 
Ilf)  x  and  z  are  not  orthogonal. 


12a)  \/94  12b)  \/296  12c)  \/80  12d)  \/90  12e)  w  and  x,  x  and  y 


13a)  wTv  =  —16 


13b)  wTw  =  54 


13c)  cos0  = 


wHv  _  16 

||W||||V||  “  ^14^54 


8-\Z21 

63 
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3 


13d)  projwv  =  w 


wrv 

WTW 


14a)  wTv  =  -33  14b)  wTw  =  50  14c)  cos 9  = 


14d)  projwV  - 


15a)  15b)  ^  15c)  15d)  15e) 

1«) 


16)x=|u-|w-iv 


17a)  v  =  ciw+c2x+c3y  17b)  w  +  x  +  y  =  v  which  is  equivalent  to  setting  all  of 
the  constants  in  part  a  equal  to  1.  Therefore,  v  is  in  the  span  {w,x,y} 

17c)  No,  they  are  linearly  dependent  because  v  depends  on  a  combination  of  the 
other  3  vectors. 


18)  No,  b  is  not  a  linear  combination  of  the  columns  of  A  because  the  linear  system 
is  inconsistent. 
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19)  Yes,  b  is  a  linear  combination  of  the  columns  of  A  because  the  linear  system  is 
consistent. 


C.  SOLUTIONS  TO  CHAPTER  III 


1.  Check  T(x  +  y)  =  T(x)  +  T(y)  and  T(cx)  =  cT(x). 


T(x  +  y)  =  T 


( 


Xi 


+ 


y  i 

2/2 


\  / 
=  T 


VL 


Xi  +2/1 
*2+2/2 


(*i  +  2/1 )  -  (x2  +  2/2) 

0 

(*i  +  2/i  +  *2  +  J/2)2 


*i  +  2/i  “  *2  -  2/2 

0 

xx  +  2/1  +  *2  +  2/2  +  2xi2/i  +  2xxx2  +  2x2j/2  +  2xxi /2  +  2t/iy2  +  2x22/i 


r(x)  +  r(y)  =  t 


VL 


Xi 

X2 


+  r 


VL 


2/1 

2/2 


Xi  —  x2 

1 

1— l 

1 

to 

_ 1 

0 

+ 

0 

(xi  +  x2)2 

_  ( 2/1  +  S/2)2  _ 

*1  +  2/1  -  X2  -  2/2 

0 

x\  +  2/1  +  *2  +  v\  +  2xx  x2  +  2i/i2/2 


Therefore,  T(x  +  y)  ^  T(x)  +  T(y). 


T(cx)  =  T 

f 

c 

X\ 

u 

CX 1 

)-[ 

.  X2 . 

J 

CX2 

)  L 

CXi  —  cx2 
(cxx  +  cx2)2 


=  c 


xx  —  x2 
c(xx  +  x2)2 
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Xi  \  X1—X2 

cT(X)  =  cT  Uc 

[  X2  \)  (zi  +  X2) 

T(cx )  /  cT(x).  Neither  rule  holds  true,  therefore,  T  is  not  a  linear  transformation. 


2)  Check  T(x  +  y)  =T(x)+T(y)  and  T(cx)  =  cT(x). 


T(x  +  y)  = 


T  11  +  Sl 


x\  +  2/1 
*2  +  y2 


3(*i  +  yi) 

2(®i  +  2/1)  -  (^2  +  2/2) 
22  +  2/2 


3xi  +  3j/i 

=  2a?i  +  2 j/i  -x2-V2 
x2  +  2/2 


/  r  -I  \  /  r  1  \  3®i  3  yi 

CEi  1  12/11 

T(x)  +  T(y)  =  T  +T  =  2xi  -  x2  +  2t/i  —  2/2 

\  x2  )  2/2  / 

' L  x2  2/2 


3*1  +  3r/x 

=  2xi  +  2yj  -  x2  -  2/2 
®2  +  2/2 

Therefore,  T(x  +  y)  =  T(x)  +  T(y). 


T(cx)  =  T  c 


2cxi  —  cx2  —  c  2xi  —  x2 


cT(x)  =  cT  I  J  =  c  2xi  —  x2 

J  x2 
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T(cx)  =  cT(x).  Both  rules  hold  true,  therefore,  T  is  a  linear  transformation. 


3)  Check  T(x  +  y)  —  T(x)  +  T( y)  and  T(cx)  =  cT(x). 


x\  +  yi 

x\  +  yi  +  4(x2  +  y2) 
(ah  +  y\)(x2  +  y2) 

x\  +  y\ 

x\  +  y\  +  4x2  +  4y2 
x\x2  +  yix2  +  xiy2  +  2/12/2 


r(x  +  y)  =  T 


X\ 

x2 


+ 


y\ 

2/2 


=  T 


\) 


xi+yi 

X2  +  V2 


r(x)  +  r(y)  =  r 

Xi 

)+r( 

2/i 

Xi 

2/i 

= 

xi  +  4x2 

+ 

2/i  +  4j/2 

\ 

. x 2 . 

/  v 

.  2/2  . 

) 

XiX2 

2/i2/2 

Xi  +yi 

xi  +  yi  +  4x2  +  4y2 
X\X2  +  2/12/2 


Therefore,  T(x  +  y)  ^  T(x)  +  T(y). 


II 

5 

c 

Xi 

W[ 

X2 

)  u 

CX\ 

Xl 

CX 1  +  4cX2 

—  c 

xi  +  4x2 

C2x \X2 

CXiX2 

X\ 

cT(x)  =  cT  f  |  Xl  )  =  c  Xl+Ax2 

XaX2 

T(cx )  7^  cT(x).  Neither  rule  holds  true,  therefore,  T  is  not  a  linear  transformation. 


' 

\ 

X\ 

.  X<1  - 

/ 
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4a)  || A||,  =  14  4b)  HBII,  =  14  4c)  ||  C  ||,=  13 


5a)  ||A||„  =  13  5b)  ||B||„  =  18  5c)  ||  C  ||«,=  10 

6a)  elementary  matrix  (multiplies  row2  by  —4) 

6b)  not  an  elementary  matrix 

6c)  elementary  matrix  (interchanges  row2  and  row3) 

6d)  elementary  matrix  (replaces  rowl  by  — 3row3  +  rowl) 

6e)  not  an  elementary  matrix 

6f)  elementary  matrix  (multiplies  rowl  by  —2) 


7a)  Ei  interchanges  rows  1  and  3  of  a  compatibly-sized  matrix.  E2  replaces  rowl  of  a 
compatibly-sized  matrix  by  (row2  —  rowl).  E3  multiplies  row2  of  a  compatibly-sized 
matrix  by  —2. 


0  0  1 

0  1  -1 

7b)  E1E2E3  — 

0-20 

7c)  E3E2E1  — 

0-2  0 

-1  -2  0 

1  0  0 

7d)  No,  because  matrix  multiplication  is  generally  not  commutative. 


8a)  Independent  8b)  Dependent  8c)  4  8d)  4  8e)  4 


9a)  rank(A)  =  2  9b)  x  =  [4 r  +  28s  +  37 1  -  13u,  2 r  +  12s  +  16 1  -  5 u,  r,  s,  t,  u)T 
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D.  SOLUTIONS  TO  CHAPTER  IV 


la)  linear  lb)  nonlinear  (xy)  lc)  nonlinear  (x2) 

Id)  linear  le)  nonlinear  (3y-1)  If)  linear  lg)  nonlinear  (sin(x)) 


2a)  x  = 


2b)  x  = 


2c)  x  = 


8  2  4 


T 


1 — 1 

to 

h-i 

GO 

_ 1 

-6 

3a) 

2  3  4  |  4 

3b)  x  = 

4 

4  5  9  |  5 

1 

4a) 


0-2  3 

3  6-3 

0-6  9 


1 

-2 

9 


4b)  No  solution.  System  is  inconsistent. 


1 

-1 

2 

-1 

-1 

t  -  1 

2 

1 

-2 

-2 

-2 

2s 

5a) 

-1 

2 

-4 

1  1 

1 

5b) 

s 

3 

0 

0 

-3  | 

-3 

t 

where  s,  t  €  TZ 
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1  2  1  |  6 

8a)  1  2  2  |  7  8b)  No  solution.  System  is  inconsistent. 

2  4  2  |  15 


1  2  1  |  bx 

10a)  2  3  2  |  b2 

4  5  8  |  63 


10c)  No  lOd)  No 
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1 

-2 

-1 

1  h 

-56i-265-3t 

3 

11a) 

-4 

5 

2 

1  b2 

lib) 

—4b\  —62  —2 1 

3 

,  where  t  €77 

4 

7 

8 

1  h 

t 

11c)  15&i  +  562  0  lid)  b  — 


2 1 

'  3 


,  where  t  6  77. 


12a)  The  system  has  an  infinite  number  of  solutions  and  is  therefore,  consistent. 

i  t 


12b) 


-iio-a»±5ot  2 +  -29 s  +  t  -9s  4°-2110s-30f  s  t 


,  where  s,t  €77. 


13a)  A"1  = 


2  -1 
-5  3 


13b)  Nonsingular 


14a)  A'1  = 


3  6 
1  2 


14b)  Singular 


15a)  A 


-1  _ 


0  1  -1 

2  -2  -1 

-1  1  1 


15b)  Nonsingular 


16a)  A  1  = 


-5 

0  2 


-12 

9 


16b)  Nonsingular 
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17a)  A-1  does  not  exist.  17b)  Singular 


0  10  2 

1-1-2  2 

18a)  A”1  = 

0  13-3 

-2  2  3-2 


19a)  A-1  does  not  exist.  19b)  Singular 
i  2  1 

20a)  A-1  =  20b)  Nonsingular 

1  —i 

i+i  o 

21a)  A-1  =  2  21b)  Nonsingular 

0 


i+i  i±i  i 

2  2  2 

22a)  A-1  =  =d  22b)  Nonsingular 

— i-H  i±j  -i 

2  2  2 


1  o  0 

23)  A"1  =  0  \  0 

0  0  1 


18b)  Nonsingular 
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24)  A”1  = 


I  A 

4  20 


0  0 


33 

20 

1 

5 

~1 


25)  Gaussian  elimination  will  produce  a  row  of  zeros. 


26)  Gaussian  elimination  will  produce  a  row  of  zeros. 


28)  ni,i&i,i  +  fli, 2^2,1  —  1  and  #2, 1^1,1  +  #2, 2^2,1  —  0  — >  62,1  — 
Substituting  62,1  into  the  first  equation  yields 


_  t  ,  _  -02,1^1,1  _  ■, 

Ol,l0l,l  +  &1,2 -  —  1 

02,2 


*1.1  = 


®2,2 


Ol, 1^2,2  ~  02,101,2 


Substituting  &i,i  into  62,i  =  —  °2^'1  yields 


&2,1  = 


02,1 


01,1^2,2  —  02,iai,2 


02,1^1,2  +  02,2^2,2  =  1  &ud  Oi,  161,2  +  Oi,2&2,2  —  0  - *  &2,2  — 

Substituting  62,2  into  the  first  equation  yields 


02,1^1,2  +  02,2 


—Qi, 1*1,2 

Ol,2 


U,2 


-02,1 


Ol, 1*2,2  ~  02,lOi,2 


Substituting  b1>2  into  62, 2  =  yields 


Lfl12  ~ 

01,1 

&2,2 

-  02,101,2 

1 

0 

0 

3  - 

-6 

-3 

29)  L  = 

2 

3 

1 

0 

and  U  = 

0 

4 

8 

4 

3 

_  1 

4 

1 

0 

0 

2 
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30)  L  = 


1 

0 

0 

0 

4 

3 

2 

1 

2 

1 

0 

0 

and  U  = 

0 

3 

2 

1 

1 

2 

1 

0 

0 

0 

2 

1 

2 

1 

2 

1 

0 

0 

0 

1 

1 

2 

5 

31)  z  = 

4 

and  x  = 

1 

2 

2 

4 

5 

1  0  0 

5  5  10 

-1 

32)  L  = 

-f  i  o 

,u  = 

0  1  7 

,  and  x  = 

1 

0  4  1 

0  0  -2 

0 

1  0  0 

-1  -3  -4 

33)  L  = 

-3  1  0 

,u  = 

0  1  -22 

,  and  x  = 

2  2  1 

0  0  63 

1  0  0 

4  -2  1 

34)  L  = 

-3  1  0 

,u  = 

0-3  3 

»  x  = 

2  -1  1 

0  0-1 
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